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PREFACE 



This research project represents both a culmination and a beginnine 
As a culmination, it records activities and ideas examined over a per- 
iod of years with a variety of people; as a beginning, it provides the 
base for research and trailing activities that can be broad in scope 
* 4^1 implications. Ihe materials developed have much poten- 

tial for training of administrators and for manipulation of variables 
in other experiments. The results gene»*ate specific recommendations 
for the practice of selection of teacheis, and at the same time will 

allow future experiments to control variations due to information for- 
mat. 



A number of people have contributed to this research. Some have 
contributed ideas, others technical assistance, some the labor necessary 
to produce materials and communications, and still others the inspira- 
tion and encouragement necessary for a sustained project. 

For several years I have been intrigued with the decision proces- 
ses of administrators, partly because of the centrality of these proces- 
ses to the major functions of administration and partly because of the 
potential generalizability of these processes to other cognitive tasks. 
Likewise, I have been interested in the teacher selection process for 
two major reasons: (a) it provides an opportunity for an educational 

administrator to make a major contribution to the improvement of a 
school system, and (b) it affords an example of the decision process 
in which the process Itself can be studied systematically. 

The summer of 1965 presented an opportunity to focus on these two 
areas of Interest (i.e., decision making and teacher selection) and to 
design the basic study described in this report. During that summer, 

^ P’fosram co-sponsored by the University of W1 scons ini and 
the U. S. Office of Education. The program was initiated to Increase 
research skills and to provide assistance in the design of experiments. 
Several participants in the summer program provided assistance with the 
design of the experiment, and each of the staff members (Julian Stanley, 
Director; Richard Schutz, Gene Glass, and Frank Baker) reacted to the 
research proposal and assisted with the conceptual aspects of the design. 
Four graduate assistants to Dr. Stanley (Adrian VonMonfrans, Andrew 
Porter, Tom Houston, and Alan Abrams) also reacted to the proposal. 

Dr. Glenn Boerrigter, of the U. S. Office of Education, was a partici- 
pant-observer in the summer program and gave helpful suggestions regard- 
ing the writing of the final proposal. 

The project formally began in September of 1966, and has been 
conducted with the benefit of three excellent research assistants. During 
the materlals-development phase, major responsibilities were delegated 

Bauthues— written documents and data collection devices; 
mke Hickey— filmed Interviews; and Dale Palmer— material for simulating 
the situation. Each of these research assistants continued to work on 
t e project throughout the experiment, data analysis, and writing phases 
but in less specialized roles. 
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During the data analysis phase, Dr. Gilbert Sax provided consulta- 
tion and assistance. One of his able graduate students, LaVerne Collet, 
assisted with statistical analysis and computer programming. 

An advisory committee consisting of~ 

Mr. Lee Blakeley, Assistant Superintendent in Charge of 

Instructional Services, Northshore School District No. 417 

Dr. Wendell French, Professor, College of Business Administra- 
tion, University of Washington 

Dr. Frederic T. Giles, Dean, College of Education, University 
of Washington 

Dr. Gerald Hester, Superintendent, Vashon Island School 
District No. 402 

Mr. S. Lyman Hilby, Manager School & College Placement, 

College of Education, University of Washington 

Dr. Jack E. Kittell, Associate Professor, College of Education, 
University of Washington 

Mr. Clayton Knittell, Principal, Central Elementary School, 
Snohomish School District No. 201 

Dr. Gilbert Sax, Professor, College of Education, University 
of Washington 

Mr. Kenneth H. Storaasli, Director of Personnel, Clover Park 
School District No. 400 

provided reactions to materials and ideas during the first year of the pro 
ject. Nine school districts enthusiastically allowed all of their elemen 
tary principals to contribute one day as subjects for the experiment. 

Secretarial assistance was provided by Mrs. Shirley Doepkin during 
the first year and a half and by Mrs. Mary Weis in the final stages. 

It has been possible to obtain from many people their reactions 
to the ideas presented in the project. One paper was read at the 1967 
AERA meeting) a symposium consisting of four papers was presented at 
the 1968 AERA meeting, a presentation was made to graduate students at 
the University of Nebraska in the summer of 1967, a report was made to 
the 1967 meeting of the National Conference for Professors of Educational 
Administration, three oral reports were given to subjects of the experi- 
ment, the materials were presented to members of the Northwest School 
Personnel Association, and several classes in educational administration 
at the University of Washington were exposed to the materials used and 
the results of the experiment. In virtually all of these situations, 
the helpful reactions obtained have contributed to the conduct of the 
project or the interpretation of the results. 



Dale L. Bolton, August, 1968 



CHAPTER I. SUMMARY 



This summary presents a brief description of the problem that was 
investigated, its scope, the objectives of the study, the methods used, 
the results obtained, and the implications of the project. It also 
presents some recommendations for practice and further research. 

The Problem and Its Scope 

This research project was conducted to determine whether the format 
of information affects decisions made in the selection of teachers. To 
make this determination, it was necessary to simulate an educational 
situation in order to manipulate and control variables. Consequently, 
the general purposes of the study were: 

1. To develop means for simulating a teacher selection situation 
within which administrative-decision-making can be studied. 

2. To determine the effects of four variables, all related to 
information -format, on teacher selection decisions in relation 
to the consistency of the decisions, the fineness of the dis- 
criminations made, the time needed to make the decisions, and 
the confidence that the administrator has in his decisions. 

The four independent variables manipulated were: (a) amount of instruc- 

tion provided on how to process information, (b) number of written docu* 
ments presented, (c) degree of masking of information, and (d) interview 
information. The variables involved in the study are shown in the follow- 
ing table: 



Table 1.1 

Variables Involved in the Study 



Independent Variables 


Dependent Variables 


Variables 




Levels 


1. Instruction 


d • 


Instruction 


1. 


Time (total time taken) 




b. 


No Instruction 


2. 


Discrimination (by grouping. 


2. Documents 


d • 


Multiple 




and by estimated consequences) 




b. 


Single 


3. 


Certainty (regarding ranking. 


3. Masking 


d • 


Considerable 




and regarding estimated conse- 




b. 


Partial 




quences) 




c • 


None 


4. 


Consistency (regarding ranking. 


4. Interview 


a. 


Audiovisual 




and regarding estimated conse- 






(filmed) 




quences) 




b. 


Audio (tape 
recorded) 








c • 


None 







The general significance of the teacher selection problem is par- 
tially determined by the number of teachers selected annually in the 
United States, and partially by the fact that each teacher represents 
a potential gain or loss to the school system in terms of goal accom- 
plishment, The specific significance of this study is determined by 
the particular dependent variables chosen for study, and by the fact 
that teacher delection decisions are based on information and are af* 
fected by the format of that information. 

One of the dependent variables, viz . . time needed to make a deci- 
sion, is a practical consideration in decision making and is independent 
of the "goodness" of the decision made. The importance of the time 
factor seems obvious, in that a small amount of time saved on each of 
a large number of teacher selection decisions means a considerable sav- 
ing to the school district. 

The significance of the discrimination variable is based on the 
view that fine discriminations are needed to make subtle distinctions 
among teachers where the potential loss due to error is high and where 
the range of applicants is relatively narrow. In addition, consistency 
in the prediction of outcomes of selection decisions helps to maximize 
long-range goals by reducing the discrepancy between predictions and 
actual outcomes. If selection decisions are not reliable, relative 
losses will occur to the school system because of unaccomplished goals 
and purposes. (This is not to be construed as an argument against 
variability within a teaching faculty; if variability is a desired 
outcome, it should be planned for and predicted in the same manner as 
any other desired outcome.) 

Unless decisions are both discriminative and consistent, there is 
little foundation upon which to accumulate evidence as to the validity 
of the decisions being made. There is no basis for using the outcomes 
of past decisions to improve future decisions; hence, decision making 
is likely to remain a vague, intuitive process. 

Another practical factor related to the significance of the problem 
is the confidence or certainty of the decision maker regarding his 
decisions. Its significance is based on the view that decisiveness 
in an administrator is a good quality, that uncertainty can lead to 
indecision, and that uncertainty can cause vacillation and wasted motion. 

The significance of the four dependent variables is established 
then on a direct or indirect relationship to tangible gains for a school 
district. The question of the validity of the decisions, or the "good- 
ness" of the decisions in terms of whether the "correct" teacher is 
selected has been omitted intentionally from this study. It is assumed 
that local school systems define teacher effectiveness according to 
specified local criteria; if so, the local system will be able to 
specify the outcomes desired in terms of teacher behavior. It is also 
assumed that if the decision maker can consistently discriminate among 
teacher applicants in a simulated situation, he should be able to relate 
teacher applicants consistently to selection criteria specified by a 
local school district. This last assumption is worth empirical verifi- 
cation, however, and should be studied at a later date. 
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The independent variables were chosen because of their relationship 
to the way an administrator might use his time and how information might 
be processed. The use of a single summary document or partially-masked 
information (i.e., exceptional data rather than the total data available) 
might allow clerical help or data processing equipment to transform 
information into a more useful format and therefore permit the adminis- 
trator to use his time in actual decision making rather than cumulating 
and collating information. However, giving the decision maker instruc- 
tions regarding how to process inforuiation might facilitate decisions 
to the extent that it would nullify the benefits of mechanical or clerical 
manipulation of information. If the interview information is beneficial, 
then it must be retained for the purpose of assisting in decision making. 
If, however, administrators do not benefit from interview information, 
perhaps the interview should still be retained— but for purposes other 
than decision making (for example, to provide the applicant with infor- 
mation regarding the teaching situation, to help the applicant to make 
a decision, or to begin orienting the prospective teacher regarding the 
expectations of the school organization). 

In this study, it was proposed that decisions are based on informa- 
tion, that the same information may be presented in differing formats, 
and that the format of the information may affect decisions. The general 
problem, then, was to find an information format that does not adversely 
affect teacher-selection decisions as far as: (a) amount of time needed 

to make decisions, (b) fineness of discriminations made, (c) feeling of 
certainty regarding the decisions, and (d) consistency in prediction of 
decision outcomes. 



Objectives 

It was hypothesized that four inf ormation -format variables affect 
the decisions that are made by modifying the consistency of the deci- 
sions, the fineness of the discriminations made, the time needed to make 
the decisions, and/or the certainty that the decision maker has in his 
decisions. The over -all objective was to find a maximally useful format 
(within the limits of the variables manipulated) for presenting informa- 
tion on teacher applicants. In order to accomplish this purpose, specific 
hypotheses regarding the main and interaction effects among the independ- 
ent variables were tested for each of the four dependent variables. 

Methods 

There were three stages to this research project: (a) descriptive 

and visual materials were prepared to create a simulated teacher selection 
situation, (b) an experiment was conducted within the simulated situation 
by asking subjects to make decisions regarding fictitious teacher appli- 
cants, and (c) statistical analyses were made of the subjects' responses 
to determine the effects of the experimental variables. 

The simulated situation within which the experiment was conducted 
offered the advantages of: (a) making it possible to control some of 

the variables that might ordinarily affect selection decisions (e.g., 
assignment situation, supervision situation, evaluation procedure, 
independence of decision, order of presentation of information regarding 



applicants, motivation of subjects); (b) permitting the investigator to 
manipulate systematically variables of interest; and (c) making it 
possible to maintain administrative behavior that is consistent with 
actual situations* 

Descriptive and visual materials developed included: (a) written 

documents (transcripts, credentials, etc.) needed for describing ficti- 
tious teacher applicants; (b) audio and audiovisual materials for simu- 
lating teacher interview situations; and (c) instructional materials 
for describing the hypothetical situation for which the applicants were 
considered* 



Fictitious applicants were created for the simulation of teacher 
selection. An attempt was made to create applicants whose personalities, 
characteristics, and experiences were distinct, yet similar enough to 
require a rather fine degree of discrimination to note the distinctions 
among them* The five factors of teacher behavior (for elementary 
teachers) identified by Ryans in the Teacher Characteristics Study 
(Ryans, 1960) were used for dimensions around which to build the appli- 
cants' personalities* These factors can be identified by their first- 
order dimensions as: (a) originality, (b) organization, (c) empathy, 

(d) sociability, and (e) buoyancy. The five factors were varied among 
the fictitious applicants in such a way that the personality of each 
girl was obviously high on one factor. Two other factors were less 
obvious (but present), and two others were not evident. 

The subjects used for this study were selected from three counties 
in the State of Washington. Districts were randomly chosen from these 
three counties, and the first nine were asked to participate. These 
districts had slightly more than the number of principals needed for the 
study, and 144 were randomly chosen and assigned to the 36 treatments. 
The design was a completely randomized 2x2x3x3 fixed model treatment 
arrangement with measures on all four of the dependent variables. 

The general experimental task performed by each subject was to 
examine eight fictitious applicants for a hypothetical teaching position 
and make decisions regarding thcs appropriateness of each applicant for 
the position. Each subject was asked to: (a) estimate how each appli- 

cant would be evaluated on a Teacher Evaluation Instrument (TEI) at the 
end of one year of teaching, (b) rank order the eight applicants accord- 
ing to their desirability for the hypothetical situation, (c) make a 
statement about the certainty of his judgments regarding the estimates 
on the TEI and the rank order by indicating how willing he would be to 
bet that his judgments were correct, and (d) group the eight applicants 
according to selected attributes or characteristics. 

The above tasks were completed during the morning session of the 
experiment. For purposes of measuring the consistency of the decisions, 
a retest was administered in the afternoon in the following manner. 

Five of the eight applicants presented in the first sessions were 
repeated in the afternoon session. These five applicants were made to 
appear different by modifying certain minor data, e.g*, age, birthplace, 
height, and weight. Changes in make-up, hairpieces, and clothes altered 
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appearances during the filmed interview. The other three applicants 
used during the first session were decoys and were replaced by consider- 
ably different applicants during the afternoon session. The decoys 
appeared late in the order of presentation in the first session and 
early in the second session to aid in forming the impression that the 
second set was an entirely new set of applicants. It was assumed that 
the insertion of the decoys did not affect the decisions regarding the 
five applicants on whom repeated measures were taken. 

An analysis of variance for the 2x2x3x3 factorial experiment was 
completed for the main and interaction effects of the four independent 
variables* By using the single measure of time, and two measures for 
each of the other dependent variables, there were seven ANOVAs computed. 

The simulation and the experiment were based on the premises that 
decision making in the selection of teachers involves processing of 
considerable information; determination of the general situation and the 
particular assignment for which the teacher is to be selected; analysis 
of the various applicants, their background of experience and education; 
prediction of behavior of each applicant in the position for which he 
is being considered; and judgment regarding which behavi'irs are most 
appropriate for the position. 



Results 

The analyses of variance for the experiment yielded the following 
results : 

1. Instruction on how to process information, under these experimental 
conditions, reduced the amount of time it took to make decisions. 
Instruction also interacted with documents by reducing the time 
for multiple documents and by increasing the discrimination of 

the subjects* estimates on the Teacher Evaluation Instrument. 

2. The single, summary document reduced the time it took to make 
decisions and increased the amount of discrimination in esti- 
mating applicant performance at the end of the first year of 
teaching. 

3. The degree of masking of information (i.e., whether complete, 
only "non-average," or only ‘‘exceptional" information on the 
applicants was available to the subject) reduced the time need- 
ed to make decisions. However, the degree of masking reduced 
discrimination proportionately. The degree of masking inter- 
acted with the documents variable in the following manner: 
lAien multiple documents were used the most consistency was 
obtained with considerable masking; when the single document 
was used, it yielded the most consistency with no masking* 

4. Interview information , in audiovisual form, increased the time 
needed to make decisions, increased the discrimination on esti- 
mates of consequences on the Teacher Evaluation Instrument, 
and increased both measures of certainty. There appeared to 
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be little difference between the effect of audio and audiovisual 
interview information on the time and discrimination measures. 
The interaction of interview information with documents (where 
certainty was measured by the estimation on the TEI) indicated 
that the audio information depressed the certainty scores with 
the multiple documents. 

Implications 

The implications of this research project for practice in the selec- 
tion of teachers are as follows: If principals or personnel directors 

involved in selection of teachers are of a similar nature to the subjects 
used in this study, their decisions regarding the selection of teachers 
will be affected by the format of information about applicants. Further, 
one would expect that the format that would yield optimum results (as 
far as time, discrimination, consistency, and certainty are concerned) 
would consist of instructions regarding the processing of information, 
a single summary documents no masking of information, and interviews 
that include visual as well as audio stimuli. 

The implications of this research project, resulting from the 
materials developed, are considerable. The materials used to simulate 
the decision-making process for selecting teachers provide: (a) a means 

of teaching the selection process to school administrators, and (b) 
a setting whereby further problems in decision making and teacher selec- 
tion may be studied. The simulated situation v;ill allov; administrators 
and prospective administrators to make decisions in an environment where 
the consequences will not be irretrievable, wheye comparisons of deci- 
sions can be made and discussed, and where additional trials will promote 
learning. In relation to increasing the knowledge of decision-making 
processes, an optimum information format will allow experiments to be 
conducted in a simulated situation without fear that the results will 
be adversely affected by the manner of presenting the information to 
subjects. 

TSie materials used to simulate the decision-making process for 
selecting teachers, then, provide a setting whereby descriptive and 
prescriptive theories of decision making can be tested. In addition, 
other variables that have been controlled in this experiment— especially 
those dealing with situational factors and interview information— can 
be manipulated in future experiments to determine their contribution 
to decision making. 



Recommendations 

The following recommendations are an outgrowth of the results of 
the experiment: 

1. Administrators of school districts should: 

a. Develop ways to use the format which was found to be 
optimum in this study in providing information to deci- 
sion makers who select teachers. 



b< Not hesitate to use audio interview information where it 
is necessary or expedient (for example, telephone inter- 
views with persons who are considerable distances from 
the location of employment). 

2. Programs should be devised to assist school districts to: 

a. Develop single, summary documents appropriate for local 
use. 

b. Provide instructional programs regarding the processing 
of information for selection of teachers. 

3. Additional research and development activities should be 

initiated and conducted to determine: 

a. The conditions under which individuals can learn to select 
teachers for specific selection criteria representing the 
value system of a specified school district. The materials 
developed by this project should be useful in such a devel- 
mental program, but they should be field tested for their 
instructional utility. 

b. How differing value systems of individuals interact with 
the abilities of the individuals to estimate consequences 
of alternatives. The simulated materials developed during 
this project should assist the study of such interaction. 

c. How certain variables controlled in this experiment (e.g., 
grade level of the vacancy, sex of the applicants, age 
range of the applicants, racial and ethnic background of 
applicants) affect the decision making of administrators. 

d. The effect of the instructional variables on other popu- 
lations and with varying amounts and types of instruction. 

e. Vlhy the interaction between degree of masking and single 
documents resulted in reduced consistency. Also, there is 
a need to know why masking reduced discrimination. 

f. Whether the lack of difference between the audio->interview 
information and the audiovisual-interview information is 
consistent under other circumstances. For example, if 
instruction in specific techniques of observation and listen- 
ing skills are provided, will differential results occur? 

g. Why the nature of the Interview information affects differ- 
entially the certainty of decisions made with different 
types of documents. 
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CHAPTER II. INTRODUCTION 



Background for the Study 

§ The decision to select a teacher from among a number of applicants 

Is the culmination of series of preliminary decisions which constitute 
the selection process. So crucial is the selection of a teacher to the 
quality of the educational program that it seems obvious that this deci- 
sion should be made only with the utmost certainty regarding its utility. 
Yet» this is an anomaly of the selection process in education — and of 
selection in general — • that such decisions are frequently intuitive and 
arbitrary. Contemporary administrative literature contains many theories 
of widely diverging complexity and quality which purport to improve the 
selection process. However, the fact remains that very little empirical 
data exist to either substantiate or disprove these theories; and conse- 
quently, the process remains, in many respects at least, a highly sub- 
jective one. 

The purpose of this research was twofold: (a) to develop a means 

for simulating a teacher selection situation within which administrative 
decision making could be studied; and (b) to determine the effects of 
four information-format variables on teacher-selection decisions in 
relation to the consistency of decisions, the fineness of discrimina- 
tions made, the time required to make decisions, and the feeling of 
* certainty that the decision maker has in his decision. 

Although there are several ways in which the study of the decision - 
f making process of educational administrators can be approached, no others 

offer the advantages of the simulated situation which permits the inves- 
tigator to control and manipulate variables and at the same time to 
maintain administrative behavior that is consistent with actual situa- 
tions. Therefore, materials were developed for simulating an educational 
situation. 

The format of data regarding teacher applicants in the simulated 
selection situation was modified and manipulated by means of four inde- 
pendent variables: (a) the number of written documents presented; (b) 

the amount of instruction provided decision makers regarding how to 
process information; (c) the degree of masking of information; and (d) 
the format of interview information. 

A number of measurement dimensions could have been considered for 
determining the effect of the four independent variables, but this study 
was concerned with determining their effect on the following: (a) the 

consistency in estimating outcomes of decisions, (b) the fineness of 
discriminations made, (c) the amount of time needed to make decisions, 
and (d) the feeling of certainty regarding the decisions. 

The variables involved in the study and the levels of each variable 
are shown in Table 2.1. 
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Table 2.1 



Variables Involved in the Study 



Independent Variables 



Dependent Variables 



Variables 



Levels 



1 . 

2 . 

3. 

4. 



Instruction 


A • 


Instruction 




b. 


No Instruction 


Documents 


A • 


Multiple 




b. 


Single 


Masking 


a. 


Considerable 




b. 


Partial 




c* 


None 


Interview 


a. 


Audiovisual 






(filmed) 




b. 


Audio (tape 






recorded) 




c • 


None 



Time (total time taken) 

Discrimination (by groupings, 
and by estimated consequences) 

Certainty (regarding ranking, 
and regarding estimated con- 
sequences) 

Consistency (regarding rank- 
ing; and estimated consequen- 
ces) 



The problem vhich was the focal concern of this study was therefore 
one of determining conditions whereby administrators who select teachers 
can be consistent in the prediction of consequences of their decisions, 
can differentiate among teachers who are very nearly alike, can make 
decisions rapidly, and can be relatively certain of the decisions they 
make. 



Significance of the Variables 

The independent variables were chosen because of their relationship 
to the way an administrator might use his time and how information might 
be processed. The use of a single summary document or partially-masked 
information (i.e., exceptional data rather than the total data available) 
might allow clerical help or data processing equipment to transform 
information into a more useful format and therefore permit the admini- 
strator to use his time in actual decision making rather than cumulating 
and collating information. However, giving the decision maker instruc- 
tions regarding how to process information might facilitate decisions to 
the extent that it would nullify the benefits of mechanical or clerical 
manipulation of information. If the interview information is beneficial, 
then it must be retained for the purpose of assisting in decision making. 
If, however, administrators do not benefit from interview information, 
perhaps the interview should still be retained — - but for purposes 
other than decision making (for example, to provide the applicant with 
information regarding the teaching situation, to help the applicant to 
make a decision, or to begin orienting the prospective teacher regarding 
the expectations of the school organization). 
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The more important decisions made by educational administrators are 
those concerning the selection of teachers. Each teacher represents a 
potential gain or loss to the school system in terms of goal accomplish- 
ments. The four dependent variables were selected because of their 
potential tangible gains for a school district. One of them Is concerned 
with the reliability of decisions* (l.e., the consistency with which pre- 
determined organizational outcomes are predicted). Unless outcomes of 
decisions are consistently predicted, long-range goals will not be maxi- 
mized because of the descrepancy between prediction and the actual out- 
come. Therefore, if selection decisions are not reliable, relative los- 
ses will occur to the school system because of the unaccomplished goals 
and purposes. (This is not to be construed as an argument against vari- 
ability within a teaching faculty; if variability Is a desired outcome. 

It should be planned for and predicted In the same manner as any other 
desired outcome.) 

The need for making fine discriminations among teacher applicants 
Is probably not as obvious as the need for consistency. However, as the 
quality of teacher appli<^ants Improves, and as pressures Increase for 
minimum error In selection, the need to make subtle distinctions among 
teachers who are relatively homogeneous becomes more critical. 

The reason for Interest In the time needed to make decisions seems 
obvlou% In that a small amount of time saved on a large number of teacher- 
selection decisions means a considerable saving of valuable administrator 
time for a school district. 

It Is much more difficult to relate certainty regarding decisions 
to such tangible measures as costs, but the Interest In this dependent 
variable Is due to the Idea that decisiveness In an administrator Is 
a good quality, that uncertainty can lead to lndecl|iSlon, vacillation, 
and wasted motion. 

The significance of the four dependent variables Is established 
then on a direct or Indirect relationship to tangible gains for a school 
district. The question of the validity of the decisions, or the "good- 
ness" of the decisions In terms of whether the "correct" teacher was 
selected, was omitted Intentionally from this study. It Is assumed that 
local school systems define teacher effectlvenesss according to speci- 
fied local criteria; If so, the local system will be able to specify the 
outcomes desired In terms of teacher behavior. It It also assumed that 
If the decision maker can consistently discriminate among teacher appli- 
cants In a simulated situation, he should be able to consistently relate 
teacher applicants to criteria for selection specified by a local school 
district. This last assumption is worthy of empirical verification, 
however, and should be studied at a later date. 

Because the experiment was conducted in a simulated situation, 
some of the variables that might ordinarily affect administrative deci- 
sions could be controlled (e.g., assignment situation, supervision situ- 
ation, evaluation procedure, lndepei.dence of decisions, physical conditions, 
time of year for the decision, order of presentation of subjects, order 
of presentation of information regarding applicants, and the motivation 
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of the subjects— see page 27 of Chapter III for more complete explanation 
of these variables)* 



Related Research 

In 1929, Elwood Cubberly indicated the Importance of selecting good 
teachers when he stated, "...the most Important place to guard the teach- 
ing service from deterioration is at the entrance gate." (1929:304). 
Cubberley proposed a systematic rating of teacher applicants, "...by 
which he (the superintendent) can defend his selections should they be 
called into question. Certain elements should enter Into the formula- 
tion of judgments, and such should be given proper weight." (1929:313). 

In spite of this advice, in 1958 McIntyre statedy "The literature, of 
course, is not entirely silent on the subject of teacher selection, but 
the person who is looking for selection devices that have been validated 
against accepted criteria might as well abandon the search. There are 
none." (1958:250). McIntyre's 1958 statement is just as pertinent ten 
years later. 

What explanation can be given for the 40-year void in development 
of selection practices for employing teachers? Perhaps it is due to a 
view that "The rejection of objective devices and the avoidance of mathe- 
matical treatment are rationalized on the grounds that human behavior is 
too complex to be handled by other than implicit, intuitive means." 

(Stone and Kendall : 1956: 144). 

This study was based on the view that progress in the decision and 
selection process can be made only by objective examination of human 
behavior. 

Decision Theory 

During the last 20 years, there has been a growing development and 
Interest in decision theory and application of mathematical models to 
the administrative process. This concern for decision making has per- 
meated the educational administration literature also, (Griffiths, 1958, 
1959; Dill, 1965) and there appears to be little disagreement regarding 
its significance to administrative behavior. Griffiths (1958) was one 
of the first educational administrators who emphasized the importance of 
decision making as a theory of administration; two of his objectives for 
developing decision making as a theory were to provide guides to action 
and to provide a framework within which researchers could find new know- 
ledge. (1958:119). The framework has not precipitated experimental 



1. Another reason may well be the hesitancy of local school dis- 
tricts to validate their own selection devices— in hopes that research 
on a broader scope will determine acceptable selection criteria. Gage 
(1963:118) and others have pointed out the futility of the simplistic 
models that search for universal criteria, and more complex models 
(e.g.. Medley and Mitzel, 1963) have incorporated situational variables 
that act as mediators. Such models imply the necessity of local valida- 
tion of selection procedures. 
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research, however, and this is unfortunate. (The study of Hemphill, 
ejt in 1962, although partially based on a decision -making rationale, 
was a relational study that determined factors of performance of ele- 
mentary principals and then related these factors to certain attributes 
of the principals studied. There was no attempt to manipulate variables 
to determine causes of performance.) 

The formulation of the decision problem has been analyzed by many 
authors. (Bross, 1953; Chernoff and Moses, 1959). Central to decision 
theory Is a consideration of the state of nature, a prediction of con- 
sequences of various alternatives, and an attachment of values to the 
predicted consequences. Bolton (1967), elaborating on Horst's model 
(1962), depicted the selection process as consisting basically of three 
types of measures: (a) various Information categories (e.g., biographi- 

cal Information, test scores, ratings, etc.); (b) consequent behaviors 
(e.g., those behaviors exhibited by teachers In job related activities); 
and (c) a measure of total utility, or the value of the Individual to 
the operation of the systemT^ The task facing the decision maker In 
this formulation, then. Is to decide which applicants are most likely 
to make the most valuable contribution to the major goals of the organi- 
zation. This decision is' made by predicting consequent behaviors and 
attaching values to these behaviors. The prediction of consequent be- 
haviors Is made on the basis of Information collected, and the attach- 
ment of values Is based on Institutional goals. 

There has been much discussion of utility theory In the literature 
In relation to maximization of outcomes (as predicted either objectively 
or subjectively). Some authors (Slmon:1957:198) have taken the position 
that man seeks not to maximize hlr; gains, but to satisfy minimal levels 
of necessary criteria. A decision maker does this because of the limits 
on the human mind and the complexities of the world with which he must 
cope. This study was designed to Investigate variables that have promise 
for organizing some of the complexities of the administrator's world in 
such a manner that more optimal decisions may be made. 

Bnplrlcal Evidence 

Empirical investigations of decision making have been few because 
of the problem of separating the prediction of consequences from the 
value attached to the consequences (Chernoff and Moses, 1960; Cronbach 
and Gleser, 1965). Precise measurement of utility is difficult because 
utility is a function of the product of prediction of consequences and 
the values attached to these consequences '**- unless one of these factors 
Is held constant, the other cannot be determined. This Investigation 
controlled one of the primary factors of the decision process ( viz . . 
value system) while Investigating the prediction of consequences in a 
simulated administrative system. 



2. See Appendix A for a more complete description of this explana-^ 

tlon. 



Interview 



Sydlaha conducted an experiment which Indicated that empathlc pro- 
cesses tended to Introduce errors In Interviews by causing Individuals 
to "attribute characteristics to others which, In fact, the other failed 
to attribute to themselves." (1963:347) He further stated: 

"These results also argue for the practice of putting the 
decision on an explicitly acuarlal basis, rather than leaving 
It to the 'intuition' or 'common sense' of the Interviewer, 

In which case the decision making cues are unspecified, un- 
known, or specific to the Interviewer. To the extent that 
tendencies to empathize occur, then Inter -Interviewer Incon- 
sistencies are bound to result. Effective decision making In 
selection can be achieved only through the explicit delinea- 
tion and combination of reliable and valid predictors." 

(1963:348) 

This study was designed to determine whether administrators use In- 
formation obtained In an Interview after being provided with written 
documents. If they do not, Sydlaha' s argument for actuarial decisions 
would be further strengthened and the Interview could be used for other 
purposes (e.g., to orient teacher to the district). 

General Strategy 

The views of Oskar Morgenstern are applicable to this study. 

" At all levels of decision making, facts have to be known. 

If they are lacking, efforts must be made to obtain them. 

Often they are very expensive. .. .If possible, facts should be 
qualitative; they should be precise, reliable.... There 
should not be more facts than the decision maker can handle. 

He should not be swamped by Irrelevant details. And he 
should understand the facts he gets." (1959:224) 



Morgenstern was discussing decision making In general; however, the 
Ideas are applicable to decision making In educational administration — 
and are particularly relevant to the Independent and dependent variables 
In this study of teacher selection. 

Objectives of the Study 

It was hypothesized that four Information -format variables affect 
teacher -selection decisions by modifying the consistency of the deci- 
sions, the fineness of the discriminations made, the time needed to make 
the decision, and the certainty that the decision maker expresses regard- 
ing his decision. The overall objective was to find a maximally useful 
format (wltnln the limits of the variables manipulated) for presenting 
Information on teacher applicants. In order to accomplish this purpose, 
specific hypotheses regarding the main and Interaction effects among the 
Independent variables were tested for each of the four dependent varl~ 
ables. 
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Summary 



This research was based on the notions that: (a) the selection of 

teachers Is an Important administrative decision, and both the selection 
process and decision making are worthy of empirical study; (b) the selec- 
tion decision Itself Is based on Information, and the format of that 
Information may affect the decision; and (c) the study of selection 
decisions can occur more accurately and systematically In a simulated 
situation than In a natural setting. The general problem was one of 
determining an Information format that does not adversely affect the 
teacher-selection decision. The Information-format variables were: 

(a) number of written documents presented, (b) amount of Instruction 
provided decision makers regarding how to process Information, (c) the 
degree of masking of Information, and (d) the format of Interview Infor- 
mation. The aspects of the decisions that were measured to determine 
whether they were affected by the Information format were: (a) the con- 

sistency In prediction of outcomes of decisions, (b) the fineness of 
discriminations made, (c) the amount of time needed to make decisions, 
and (d) the feeling of certainty regarding the decisions. 

Research over the past 40 years has not developed adequate selection 
procedures, possibly because many people believe human behavior Is too 
complex for objective examination. However, much work on decision theory 
has been done during the past 20 years and some empirical work has been 
reported on decision making and problems In the use of Interviews. Some 
authors have emphasized the necessity for the decision maker to under- 
stand and be able to process the Information he obtains. The general 
strategy of the research reported here Is compatible with this notion of 
Information-processing capability In decision makers. 



CHAPTER III. METHOD AND DESIGN OF THE EXPERIMENT 



Procedures 

The study Included three stages: (a) descriptive and visual mater- 

ials were prepared to create a simulated teacher selection situation; (b) 
an experiment was conducted within the simulated situation by asking sub- 
jects to make decisions regarding teacher applicants; and (c) statistical 
analyses (and interpretations of the analyses) were made of the subjects' 
responses to determine the effects of the experimental variables. 

Experimental Materials Developed 

Five types of descriptive and visual teacher selection materials 
were prepared for the study: (a) Instructional materials for describing 

the hypothetical situation for which the applicants were considered; 

(b) audio and audiovisual materials for simulating teacher interviews; 

(c) written documents (transcripts, credentials, etc.) needed for des- 
cribing fictitious teacher applicants; (d) materials for presenting one 
of the Independent variables, viz . . instructions on how to process infor- 
mation; and (e) written materials needed to obtain responses from subjects 
regarding applicants. 

In general, the content of each set of materials was determined 
after a systematic review of pertinent literature which helped to struc- 
ture a sequence of interviews with school superintendents, university 
placement directors, and school personnel directors. The ideas obtained 
from these people were synthesized and presented to a Review Panel con- 
sisting of school personnel administrators, and professors of personnel, 
elementary education, research methodology, and higher education. Revi- 
sions for modification and Improvement were made, based on the suggestions 
of this panel. The materials were then pretested on a selected group 
of graduate students, and further modifications were made based on the 
information received from the pretest group. 

Design and development of the simulated situation . On the assump- 
tion that selection decisions are mediated by situational variables, it 
was decided that relevant situational variables would be controlled in 
the experiment. More specifically the purposes of providing a complete 
description of the hypothetical situation were to: (a) remove each sub- 

ject from his own situation and place him in a controlled situation, and 
(b) allow each subject to determine criteria he considered appropriate 
for selection in the given situation. 

A determination of the situational context was made following a 
review of the literature and discussions with the advisory panel des- 
cribed above. Constraints of time for displaying the situation and the 
desirability of having the subjects retain adequate information regarding 
the situation necessitated a concise description of the factors considered 
most relevant which would satisfy the stated purposes. The situational 
presentation was divided into five sections. Hie relative Importance 
of each section to the decisions to be made and the percentage of time 
to be devoted to each section were determined by a concensus of the 
research team (the director plus three research assistants) and the 
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advisory panel. The outline below Indicates the situational content and 
the approximate allocation of time per section. 

SECTION ONE - LOCATION AND PHYSICAL SETTING OF THE COMMUNITY (57.) 

I. Location 

II. Geographic Features 

III. Areas of Districts In the Community 

IV. Population 

SECTION TWO - NATURE OF THE COMMUNITY (207.) 

I. History 

II. Recent Changes 

III. Growth Characteristics 

IV. Effects of Growth 

V. Community Leadership 

VI. Religious Groups and Activities 

VII. Recreational Facilities 

VIII. Health and Safety Agencies 

SECTION THREE - COMMUNITY SCHOOL DISTRICT (207.) 

I. History of District 

II. Local School Board Characteristics 

III. Central Administrative Structure 

IV. Physical Facilities 

V. Student Body Characteristics 

VI. Educational Program 

VII. Financial Support 

VIII. Future Needs 

SECTION FOUR - THE ELEMENTARY SCHOOL (257.) 

I. Attendance Area (Including pupil characteristics) 

II. Physical Plant 

III. Instructional Staff 

A. Age range 

B. Experience range 

C. Number 

D. Staff characteristics 

IV. Principal 

V. Primary Program 

VI. Intermediate Program 

SECTION FIVE - THE VACANCY (307.) 

I. Fourth'Grade Staff Characteristics 

II. Pupil Grouping Characteristics 

III. Supportive Personnel 

A. Professional 

B. Paraprofesslonal 

IV. Expectations of Fourth-Grade Positions 

V. Room Description 

VI. Pupil Progress Reporting 

VII. Instructional Program 
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The eituatlonal description was then created for a hypothetical 
coomunity, Norwest, using the outline as a guide. Following the com- 
pletion of a written description of the community, the manner in which 
the situation could be displayed to subjects in the experiment was con- 
sidered as a major problem because of the time and retention constraints. 
To satisfy the requirements of time and retention, and in order to gain 
an assurance of uniform presentation, it was decided an audio description 
would be tape recorded and 2x2 color slides would accompany the descrip- 
tion.^ The locations to be photographed for the slides were carefully 
selected to Insure anonymity of the actual locale and to preclude identi- 
fication of the subjects with the actual, rather than the hypothetical, 
situation. Ihe subjects were advised at the beginning of the presenta- 
tion that the district was fictitious, even though they might recognize 
several of the locales of the slides. Also, this idea was reinforced 
at the conclusion of the presentation to avoid preoccupation on the part 
of the subjects with determining where the pictures were actually taken. 
This situational presentation deviated from the usual taped commentary 
and slide combination in that the commentary content was considered to 
be the more essential element of the combination and the visual portion 
of secondary Importance. The commentary was thus presented continuously, 
and the slides were shown at appropriate Intervals. 



Since one of the controlled requirements of the experiment was an 
adequate amount of retained information regarding the hypothetical situ- 
ation, some method was necessary to test the knowledge of the situation 
as presented via the taped commentary and slides. Isolated testing, 
i.e., without some provision for feedback and reinforcement, seemed 
inappropriate for assuring an adequate degree of retention. A programed 
text^ utilizing the branching technique was devised which presented 
essentially the same content as the taped commentary. The text was 
structured to provide immediate feedback to the selected response and to 
allow the subjects to move through the program with minimum effort if 
correct responses were selected. Where responses were Incorrect, addi- 
tional information was provided via the branching technique. On comple- 
tion of the two tasks, viz . . listening to the taped commentary while 
viewing the accompanying slides and then completing the programed text, 
it was assumed the subjects were sufficiently oriented to the situation 
to be able to develop criteria as a basis for subsequent selection 
decisions. 



Before the situation was presented, the subjects were Instructed 
to make certain assumptions concerning the situation and subsequent 
tasks of the experiment. The assumptions were: 



1. Appendix B provides additional Information regarding the con- 
tent of the description and the nature of the slides. 

2. Information regarding content of the text is provided in 
Appendix C. 



1* Each subject was to assume the role of an administrator in the 
hypothetical school district, charged with he responsibility 
of selecting a replacement for a departing fourth-grade teacher 
in a particular school. 

2. The time of the year was fixed at April 15, and the subjects 
were to select a person to begin teaching next September. 

3. Those subjects receiving interview information were to assume 
that a screening interview was held prior to April 15; however, 
no information from the screening interview was available to 
the subjects. 

4. All the applicants were graduates of the same university. 

5. All the applicants had met at least the minimum state certifi- 
cation requirements and had at least provisional certificates. 

These assumptions, as well as the situational presentations, were 
held as constant factors for all subjects in the experiment. 

Development of simulated interviews . One of the independent vari- 
ables, format of interview information, consisted of three levels: (a) 

no interview information, (b) only the audio portion of interview infor- 
mation, and (c) the full audio and visual interview information. Factors 
of practicality and control precluded the use of a "live" interview 
situation in which the subjects actually interviewed applicants for the 
vacancy. Consequently, it was decided to utilize a sound and color film 
of part of an interview with each applicant for the audiovisual treatment, 
while only the soundtrack of this film was used for the audio treatment. 
The film was produced in such a manner that it focused on the applicant 
throughout the entire interview segment, and the individual who was con- 
ducting the interview remained anonymous --^only a portion of the back of 
his head appearing in the films. 

The development of the filmed interview was subject to a number of 
constraints. First, because of time and fiscal limitations, the length of 
each interview sequence was limited to nine minutes. This made it neces- 
sary to display only the probing portion of the interview, since it was 
felt, and this was supported by the literature, that this was the most 
vital poi on. Second, because of the limitations of the simulated 
situation, all interviews were with applicants being considered for a 
specific position, which was adequately delineated by the description 
of the hypothetical situation. All applicants were assumed to be at 
least minimally qualified for employment and all had presumably passed 
an initial screening interview, although none of this initial inter- 
view information was available to the subjects. Third, as a control 
measure, all applicants were female, all were between 22 and 28 years 
old, and all were of acceptable appearance, that is, none were at 
either extreme in terms of physical appearance. These controls, it 
was felt, provided a group of applicants that was relatively homogeneous 
with respect to these classification variables; such relative homogen- 
eity was necessary to be able to test for discrimination among treat- 
ments. 
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The problem confronted in preparing these materials was how to dls- 
* play adequately the personality characteristics of the applicants In a 

nlne-mlnute segment of the Interview so as to permit them to be assessed 
and rated by the subject. A Teacher Evaluation Instrument (see pages 24** 
t 25) was developed based on behavioral and personality traits related to 

the five factors (for the elementary teacher sample) delineated In Ryans' 
study (1960), and reflecting the criteria of teaching success established 
for the hypothetical district. These five factors were then manipulated 
among the sixteen fictitious applicants In such a way that a personality 
was "created" In which each applicant was obviously high on one factor 
(e.g., originality), two other factors were less obvious, but present In 
the Interview behaviors (e.g., buoyancy and sociability), and two others 
were not evident (e.g., organization and empathy). 3 

Once It was determined what the personality of the applicants would 
"look" like, the task remained: (a) to prepare scripts which would offer 
the opportunity for display of these characteristics and permit their 
assessment, and (b) to find actresses who could adequately portray the 
fictitious applicants In the manner desired. The scripts were developed 
after a thorough review of the literature pertinent to personality and 
behavioral assessment and to selection Interviews, and a survey of person- 
nel directors In twenty districts In the Seattle metropolitan area. 
Questions were developed and categorized according to the factors which 
their responses would be likely to display. Questions were selected 
« according to the following criteria: (a) they were questions which 

would reasonably be asked In an Interview; (b) they were analytical and 
probing in nature, suggesting an extended answer; (c) they would allow 
the applicant to express the designated characteristics around which the 
fictitious character was developed. 

It was decided to write a script for both the questions and the an- 
swers for two reasons: (a) a spontaneous answer, while deslreable from 

several standpoints, might violate the constraint of time on the filming; 
and (b) a spontaneous answer, while perhaps better reflecting the actual 
personality of the actress playing the role, might not clearly present 
the behavior factor Intended to be displayed, thereby eliminating a 
source of control. 

The actresses were selected from among graduating seniors In elemen- 
tary education at the University of Washington. This group was chosen 
for several reasons: (a) they were similar to the hypothetical candidates 

In age, sex, appearance, etc.; (b) their own training and background In 
education would provide a naturalness and familiarity with the situation 
which might not be as evident In non-education students; and (c) prac- 
tically all In this group had undergone one or more actual Interviews 
and would therefore be more likely to be at ease In the filming situation. 
A total of 53 girls was Interviewed for the task and eleven were finally 
accepted for the roles In the films. 






3. See Appendix D for a complete description of the characteristics 
of the teacher applicants. 
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Ryans' five central factors (for the elementary teacher sample) 

- described above were behaviorally-def ined to be relevant to the hypothe- 

tical vacancy, and candidates for the actress jobs were assessed on these 
factors and rated until the necessary number had been selected. Each girl 
t was assessed broadly at first on all five personality and behavior factors, 

but this scanning quickly focused on the traits which were predominant 
in the girl's personality, as evidenced in the interview. The actresses 
were then selected on the basis of the extent to which they "fit" one of 
the fictitious applicants whose traits they would display in the filmed 
interview. 

Once the actresses had been selected, they were given a brief writ- 
ten description of the sort of character they were to portray. In all 
cases, this character was one intended to coincide with those personality 
factors predominant in the actress. Each was also given the written 
script and asked to read it carefully, making any changes in diction or 
expression to make the responses seem as normal as possible without chang- 
ing the essential content of the interview. IChen the script was thoroughly 
learned and rehearsed several times until a natural aura pervaded the 
interview segment. 

The rationale for the design of the interview sequence was to display 
specific behavioral and personality factors which could be assessed by the 
subjects of the experiment. Two of the dependent variables to be measured 
- were: (a) the subject's certainty about his assessment of the applicants 

as he thought each would be rated at the end of her first year of teach- 
ing on a teacher evaluation instrument; and (b) his ability to discriminate 
« between applicants on a number of personality dimensions (e.g., Ryans' 

five factors). The structure of the interview sequence was developed so 
as to stringently control the characteristics displayed by the applicants 
in the interview, while retaining the realism and spontaneity of the 
situation as far as possible. 

Subsequent use of the filmed interviews in the research project and 
related presentations indicated an overwhelmingly favorable response to 
the realism of the interview. As was expected, however, a generally 
negative response was elicited concerning the narrow focus of the inter- 
view segment, most reactors indicating that it failed to display enough 
of the "total personality" of the applicants, despite the fact that all 
of the major dimensions they were required to assess in the experiment 
were evidenced in the interview and related documents. 

Development of written documents . The written documents created 
for each of the applicants were developed in conjunction with the inter- 
view materials and were designed to support, and elaborate on, the charac- 
teristics displayed in the interview. Two of the independent variables, 
number of documents and masking of information, involved manipulation of 
information on the written documents. 

The procedure followed for development of these documents included: 
first, multiple documents were created; second, information regarding the 
applicants was devised and inserted in the multiple document forms; third, 
a single summary document was created; fourth, a strategy for transfer of 
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information from the multiple documents to the single document was devised^ 
and the information was transferred; f if thi the information on the single 
documents was masked to conform to the three“level masking variable; and 
sixth, the comparable data on the multiple documents were masked. 

The documents variable consisted of two levels: (a) multiple docu- 

ments » which are the type most commonly used by districts at present y and 
(b) a single document. In developing the multiple documents, data gather- 
ing devices used by school districts were analyzed by format and the type 
of information they gathered. On the basis of; this analysis, documents 
typical of those used by school districts were created for the experiment. 
At the same time, it was intended that the documents represent the most 
reasonable of current practices. To accomplish this the literature was 
searched for empirical evidence, and opinions were solicited from the 
Advisory Committee. 

The multiple documents consisted of: (a) a letter of application; 

(b) an application form; (c) college placement credentials which included 
general information, a report on student teaching, a summary of course- 
work including grades, and letters of recommendation from professors, 
supervising teachers, and principals; and (d) a confidential teacher refer- 
ence form from a principal or superintendent. 

The second level of the documents variables was a single or summary 
document.^ It is relatively easy to construct a single document on which 
one can summarize the Information obtained on district-initiated forms— 
application, reference form, interview form— because the categories of 
information are standardized by the use of these same forms for all appli- 
cants. The difficulty lies in making a form on which one can appropriately 
summarize the information contained on the various forms initiated outside 
the district, such as college placement credentials and letters of recom- 
mendation. In the different forms and letters a district receives there 
will be a great variation in subject content, characteristics described, 
and descriptive adjectives and phrases used. However, a logical analysis 
of the different kinds of information given and the various descriptive 
terms used indicates that many different means are being employed to 
describe the same qualities and characteristics, a conclusion supported 
by the research of Tupes and Chrlstal (1961) and Peres and Garcia (1962). 



4. Examples of multiple and single documents are provided in 
Appendix E. 

5. Eight factor analysis studies of ratings on personality charac- 
teristics have been summarized by Tupes and Chrlstal (1961). The same 
five factors of typical behavior emerged from all studies. In a study of 
625 letters of recommendation by Peres and Garcia (1962), it was found 
that applicants were not described in terms of observable behavior (l.e., 
critical ineidsnts) but rather by adjectives or generalized trait names. 
The 715 adjectives found in the letters were reduced to 170 by personnel 
people. A factor analysis of these 170 adjectives identified five factors 
much the same as those identified by Tupes and Chrlstal. 
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Therefore it seems reasonable that a single document can be constructed 
on vdilch one can appropriately summarize nearly all relevant bits of Infor- 
mation received on a candidate. 

A major guideline In creating the summary document was to minimize 
the Information categories In nominal form and maximize those In scalar 
or rating form. Of the four areas comprising the form developed — General 
Information, Background Data, Summary of College Work, and Evaluative 
Data— ‘Only In the "General Information" section were the data recorded 
In nominal form. In the other three sections. Information was recorded 
on a five-column rating grid. 

In the section on Evaluative Data an applicant could be rated on 
25 characteristics pertaining to personal qualities and teaching behaviors. 
Information contained In the credentials, student teaching report, and 
letters of recommendation was assessed and recorded on this grid. A code 
cor.slstlng of both letters and colors was used so that the persons looking 
at the grid would know the source of the original evaluations* although 
for the purposes of this experiment those subjects having the single 
document treatment would not have access to these original evaluations. 

For a school district to use the summary form. It would, or course, 
have to define each characteristic or behavior Item on the document. It 
would also have to establish a policy as to what constitutes a particular 
rating for a certain characteristic or behavior. Although Initially It 
would be considerable work, the required careful examination of hiring 
policies would most likely be very beneficial. 

The Introduction of the masking variable In this study was motivated 
by the belief that It Is time-consuming to study each bit of Information; 
and since a person has difficulty assimilating large amounts of data 
(Miller, 1956), It may be better to direct attention to only those bits 
of Information that are most discriminating. There were three levels of mask 
Ing used! (a) none; (b) partial , l.e., all the "average" data were masked 
out; and (c) considerable, l.e., all "non-exceptlonal" data were masked 
out. 



Both the single summary document and the multiple documents were 
masked, so It was necessary to provide for consistent masking of the 
Information on the two levels qf the documents variable. The summary 
document was masked first. For partial masking the central column on the 
rating grids was masked out, leaving the two columns at each extreme. For 
considerable masking, only a single column at each extreme was left un- 
masked. Masking on the multiple documents was derived from the masking 
on the summary document by utilizing records of correspondent Information 
made at the time of the Initial transfer of Information from multiple 
documents to the single document. 

The masking was done to determine the effect on decisions of allowing 
the decision maker tp attend to only the more exceptional Information. In 
practice, school districts might develop different ways of providing the 
decision maker with only exceptional data. 



Developmpnt; of the instructional variable . The instructional vari- 
able was designed to give some of the subjects information-processing 
techniques and consisted of two levels, instructions or no instructions . 
The instructions given were concise and consisted of techniques which 
were assumed to be Integral to effective and efficient information proces- 
sing. The instructions were presented by tape recording to Insure con- 
sistency of presentation, and were supplemented by visual aids and a 
written summary of the five main points. Hie subjects' comprehension of 
the instructions was ascertained by a brief multiple-choice test. The 
entire process of instruction required less than ten minutes. 

The instructions provided, and a brief statement of the rationale 
underlying the instruction, are as follows: 

1. Withhold your decision until all pertinent information has been 
considered . This delay of the decision is essential, since* some items 
of exceptional information uncovered at an early stage in the process 
may otherwise prejudice or limit the decision maker's perception of the 
applicant's total qualifications for the position. 

2. Scan all the Information available to you quickly in order to 
form a general Impression of the applicant . This process assists the 
decision maker in developing a general impression of the applicant which 
allows him to focus on details of the information more efficiently and 
effectively by associating them with the impression of the total person. 

3. Decide upon criteria for selection and attempt to group the 
information Items in clusters around the particular criterion to which 
they are relevant . Since the selection criteria provide the basis for 
the selection decision, the decision maker must be aware of what they are 
and make them an explicit part of the selection process. Once recognized, 
these criteria become a sort of organizational scheme around which the 
various items of information can be clustered according to their rele- 
vance to a particular criterion. 

4. Pay particular attention to items of exceptional Information and 
attempt to summarize the exceptional points ( both positive and negative) 
for each applicant . Since the decision maker obviously cannot consider 
every information item available on each applicant and do so effectively, 
the decision will probably be based primarily on items containing excep- 
tional information. By attending primarily to such exceptional informa- 
tion, the decision maker will most likely encounter the majority of 
information items which will Influence his decision. 

5. Analyze the Information Items in terms of the relative value 
of the criteria established and the decision to be made . Some informa- 
tion is more valuable than other information in determining the qualifica- 
tions of applicants for a teaching position. This value hierarchy must 

be established by the decision maker in terms of the qualifications he 
considers most Important for the particular teaching position and situa- 
tion. The relative value for information items will then determine the 
weighting of those items in making the decision and the amount of time 
to be spent assessing them. 
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Development of response device . The response device used by subjects 
in the experiment Included the following: (a) a cover sheet and a sheet 

with appropriate date, group (a code for treatment received), the session 
(morning or afternoon), subject number for the experiment, the time begun 
and ended, and the time elapsed; (b) a set of Instructions for estimating 
how each teacher would be evaluated on a Teacher .Evaluation Instrument 
(TEI); (c) the eight TEI's; (d) a sheet for acquiring the subject's feel- 
ing of certainty regarding his accuracy In completing the TEI's; (e) a 
sheet for rank ordering the eight applicants and for obtaining a measure 
of certainty regarding the ranking; and (f) a sheet for grouping the 
applicants according to five teacher characteristics 

The entire response device was developed and presented to the Advisory 
Panel for their, consideration and review. After acquiring their reactions, 
modifications were. made and the device was tried out on a group of selected 
graduate students. Then, final modifications were made prior to use In the 
experiment. 

One part of the response device, the TEI, required more work to 
develop than the other parts, and deserves some explanation. The TEI 
consisted basically of two parts: Part One (A-F on the TEI, Appendix F) 

consisted of observable teacher behaviors In classroom situations; and 
Part 1^70 (G-I on the TEI) consisted of supplementary teacher behaviors 
that may be observed outside of classroom situations. Both parts con- 
sisted of bl-polar adjectives separated by a seven-point scale, as Illus- 
trated: 



systematic : : : : : : disorganized 

The adjectives for Part One were selected according to a two-dimen- 
sional conception of classroom behavior: (a) the teaching act, (b) 

teacher behavior. Figure 3.1 Illustrates the two dimensions, with the 
vertical scale depicting the teaching act while the horizontal scale 
Indicates the types of teacher behaviors or characteristics (Ryans' five 
factors for the elmentary teacher sample; Ryans, 1960). Bl-polar scales 
were then developed to describe teacher behaviors for each cell of the 
grid. For example, for cell # 1 , "stimulating — dull" was considered 
appropriate for describing buoyant teacher behavior during the part of 
the teaching act that Is designed to create a motivational environment . 

For Cell #2, "patient — Impatient" was considered appropriate to describe 
empathetlc behavior during guidance and counseling acts. 

Care was taken to assure that the scales described observable be- 
havior. Once a large list of scales was devised, the scales were examined 
for redundancy and were reduced to a manageable number. The scales were 
then combined acr'^ss the horizontal dimension and were organized under the 
Teaching Act headings. No heading had more than seven scales, nor less 
than three. 



6. Examples of each of these parts are provided In Appendix F. 
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Figure 3.1 
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A similar procedure was followed for Fart Two, except that a classi- 
fication system compatible with out-of-classroom activities was developed. 
The categories for the vertical dimension were: out-of-the-classroom 

duties and professional activities, relations with other people (staff 
and parents), and school -community relations. These categories formed 
the headings for Fart Two of the TEI. 

The Experiment 

This description of the experiment Includes a description of the 
subjects, the experimental task they performed, the measures obtained, 
the experimental controls used, and the general design of the experiment. 

The subjects . The subjects used for this study were selected from 
three counties in the State of Washington, viz . . Fierce, Snohomish, and 
King. Districts were randomly chosen from these three counties and the 
first nine were asked to participate. Only districts with six or more 
elementary schools were selected, and principals with less than one year 
of administrative experience were not asked to participate. The districts 
selected had slightly more than the number of principals needed for the 
study, and 144 were randomly assigned to the 36 treatments. 
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The experimental task . After being oriented to the situation, the 
general task performed by each subject was to examine eight fictitious 
applicants for a hypothetical teaching position and make decisions regard- 
ing the appropriateness of each applicant for the position. Each subject 
was asked to: (a) estimate how each applicant would be evaluated on a 

Teacher Evalution Instrument at the end of a year of teaching; (b) rank 
order the eight applicants according to their desirability for the hypo- 
thetical situation; (c) indicate his feeling of certainty regarding the 
extlmatlons on the TEI and the ranking by indicating how willing he would 
be to bet that his judgments were correct; and (d) group the eight appli- 
cants according to selected attributes or characteristics. 

The measure obtained . On completion of these tasks, a measure was 
available for two of the Independent variables, viz . . time and certainty. 

A measure of reliability of the subjects' decisions would allow an esti- 
mate of consistency; this suggested a repetition of the task at a some- 
what later date, assuming that recall would not affect the judgments. 

Aside from the hazard Involved in such an assumption, there would also 
be a problem of mortality of subjects from one testing period to the next 
and the necessity to again ln*?truct each subject regarding the hypotheti- 
cal situation. Therefore, Instead of retesting at a later date, an immedi- 
ate retest was administered in the following manner. 

Five of the eight applicants presented in the first session were 
repeated in a second session during the afternoon. These five applicants 
were made to appear different by modifying certain minor data, e.g., age, 
birthplace, height, weight, etc. Changes in makeup, hairpieces, and 
clothes altered appearances during the filmed interview. The other three 
applicants used during the first session were decoys and were replaced by 
three considerably different applicants during the afternoon session. 

The decoys appeared late in order of presentation in the first session 
and early in the second session to aid in forming the impression that the 
second set was an entirely new set of applicants. It was assumed that 
the insertion of the decoys did not affect the decisions regarding the 
five applicants on whom repeated measures were taken. 

The success of this deception was indicated by the fact that, of 
the 48 subjects who received the audiovisual interview treatment, none 
indicated that he had recognized any of the applicants as being doubles. 

After the subjects had performed the same tasks with the second set 
of applicants, it v;as possible to obtain two measures of consistency: 

(a) a correlation between the first and second ranking of the five real 
applicants, and (b) a correlation between the first and second estimates 
of how each applicant v;ould be evaluated on the Teacher Evaluation Instru- 
ment. 



Two measures of discrimination were computed after completion of the 
experiment: (a) the average number of groupings on the attributes selected 

(e.g., the larger the number of groups, the greater the discrimination); 
and (b) the mean variance of the 16 applicant scores on each item of the 
Teacher Evaluation Instrument (e.g., the greater the variance, the more 
discriminating the individual; the smaller the variance, the less discri- 
minating). 



By using the single measure of time, and two measures for each of 
the other dependent variables, seven measures were obtained for analysis* 

Controls . In the hypothetical situation, it was necessary to control 
certain factors in order to determine the effects of the treatment variables* 
The following were held constant for all treatment combinations* 

1* Assignment situation* All teachers were considered as appli- 
cants for a fourth-grade teaching position in a hypothetical 
district, to be assigned to a given building* All applicants 
were females and were relatively homogeneous in terms of appear- 
ance, age, and experience* 

2* Supervision situation* The assumption was made that all can- 
didates would receive the same quality of supervision once they 
were placed in the situation* 

3* Evaluation procedure* It was assumed that the evaluation pro- 
cedure used at the end of the year was the same for all appli- 
cants* The outcomes desired for the simulated district were 
identified and the relative value of each was indicated* 

A* Independence of decisions* Decisions of all subjects were 

obtained independently of other subjects by administering the 
tasks in monitored situations to groups of eight subjects at 
a time, each one working in a private room* 

5* Physical conditions* The physical conditions under which all 
subjects worked were similar in all respects* 

6* Time* The simulated situation assumed that all subjects were 
making the decisions on April 15th for the school year begin- 
ning the following September* 

7* Order of presentation of applicants* Since fatigue might have 
possibly affected judgments regarding applicants, a counter- 
balancing was used to compensate for any order effects* 

8* Order of presentation of information regarding applicants* This 
was the same for all treatments and levels* 

9* Motivation of subjects* Each subject was informed that he was 
to assume the role of a member of a selection committee and that 
he was to have no specific responsibility for the teacher who 
was selected for the hypothetical situation* All subjects were 
uniformly encouraged to make the best judgment possible with 
the information provided* 

Design * The design of the experiment was a completely randomized 
2x2x3x3 fixed model treatment arrangement, with measures on four dependent 
variables* The 144 subjects were assigned randomly to each of the 36 
treatment combinations of the design, a total of four subjects per cell* 
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The usual assumptions of higher order fixed model factorial experl** 
ments were made, Including the assumption of additivity of the factorial 
effects and the experimental error, and the assumption that the error 
term Is normally distributed and Independent of the treatment variables 
(Winer, 1962, Chapter 5). For each ANOVA, all hypotheses for main effects 
and Interactions were tested and the results are reported In Chapter IV 
of this report. 

Analysis of the Data 

On completion of the experiment, the following Information was 
available on each subject: 

1. Time required for each subject to complete the decision (T). 

2. A ranking of the five applicants In set one (Rl) and set two 

(R2)» 

3. A measure of certainty regarding the ranking (Ci). 

4. An estimate of consequences of selecting each of the five appli- 
cants In set one (Ej^) and In set two (E 2 ). 

5. A measure of certainty regarding the estimate of consequences 

(C2). 

6. A grouping of the subjects on selected characteristics (G). 

Time . The data for time needed to complete the decisions were 
examined, and It was determined that It was not necessary to transform 
the data prior to analysis. An analysis of variance (ANOVA) for the 
2x2x3x3 factorial experiment was completed for the main and Interaction 
effects of the four Independent variables. 

Certainty . Separate ANOVAs were completed for each certainty measure, 
testing for both main and Interaction effects. 

Consistency . Prior to using an ANOVA for determining the consisten- 
cy of the decisions of the subjects, some derived scores were computed. 
First, a correlation (rr) was computed between Rx and Rg for each subject, 
niese correlations were transformed to Fisher's r to Z transformation and 
the Z scores used for the ANOVA. This analysis allowed statements to be 
made regarding the effects of the Independent variables on the consistency 
of decisions regarding the rankings of teacher applicants. 

Another analysis of consistency was made by computing a correlation 
(re) between Ex and E 2 for each subject, transforming the re's to Z 
scores and using the resultant Z scores for the ANOVA. This permitted 
statements regarding the effects of the treatment variables on the con- 
sistency of the decisions regarding estimations of consequences. 

Discrimination . To determine the ability of a subject to discriminate 
finely among applicants, the groupings of the teacher applicants on the 
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selected attributes (G) were used. If one subject grouped the eight 
applicants into eight groups on a given attribute, e.g., scholarship, 
and another subject used three groups, then the first subject was con- 
sidered to discriminate more finely. The average number of groupings 
on the attributes selected was used as a discrimination score for an 
ANOVA of the effects of the independent variables on fineness of discri- 
mination. 

A second measure of discrimination was obtained by computing the 
variance of the 16 applicant scores on each item of the Teacher Evalua- 
tion Instrument. The mean of these variances (i.e. ,^Si2/q, where si2 

^ is the variance of the 16 applicant scores on the i.th items 

and Q represents the number of items on the TEI) was used as a discrimin- 
ation score; the greater the variance, the more discriminating the indi- 
vidual; the smaller the variance, the less discriminating the individual. 

The discrimination scores were then used in the ANOVA, and tests of sig- 
nificance were made for the main and interaction effects. 

Other Analyses . Several post analyses were made of the data. Where 
means were compared, orthogonal comparisons were made by a Newman-Keuls 
test. Also, an orthogonal comparison was made of an Interaction effect. 

On one Interaction effect where it was desirable to make comparisons 
that were not ortohogonal, Scheff^*s test was used. 

Summary 

This study included the preparation of materials for simulating a 
teacher selection situation, the conduction of an experiment, and the 
analysis and interpretation of data collected during the experiment. 

Materials for describing the hypothetical situation were displayed 
via audio instructions accompanied by 2x2 color slides; this was supple- 
mented by a programmed text which also allowed a test of knowledge 
regarding the situation. Scripts were prepared for interviews to portray 
characteristics, and films were made in order to simulate teacher 
interview situations. Written documents, representative of those commonly 
used in school districts, were developed for providing background informa- 
tion of the fictitious applicants, and this information was also presented 
on a single summary document that was developed. In addition, materials 
were developed for instructing subjects on how to process information. 

A response device was constructed which included a measure of time taken 
to perform the experimental task, a teacher evaluation instrument for the 
hypothetical district, a form for ranking the applicants and for estimat- 
ing the degree of certainty regarding these rankings, and a measure of the 
number of groups on specified characteristics. This response device allowed 
the computation of one measure of time, two measures of certainty, two mea- 
sures of consistency, and two measures of discrimination. 

The experimental design was a completely randomized 2x2x3x3 fixed 
model treatment arrangement. The 144 elementary principals used as 
subjects for the experiment were selected from three counties in the 
State of Washington and randomly assigned to the 36 treatments. The 
experimental task included the examination of fictitious applicants for 
a hypothetical teaching position, the estimation of how each applicant 
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vould be evaluated at the end of a year of teaching, the rank ordering 
of the applicants, the expression of degree of certainty regarding 
decisions made, and the grouping of applicants according to selected 
characteristics. 

The analyses of variance for each of the seven measures available 
from the experiment provided information regarding the main and inter- 
action effects. 



CHAPTER IV. RESULTS AND FINDINGS 



The general purpose of the research project was to determine the 
effects of four information-format variables bn teacher selection deci- 
sions. The purpose of this chapter is to report the results of an ex- 
periment in a simulated situation to determine these effects. The 
results of the study are presented in terms of the effects on the four 
dependent variables (three of these variables had two measures; conse- 
quently, there were seven ANOVAs computed) and then are discussed with 
relation to the four independent variables. 1 

Results of the Study 

The total results of the analyses are presented in Table 4.1. 

Since each analysis included the same independent variables, the sources 
of variation are the same for all seven analyses. Table 4.1 indicates 
results that were significant at the .05 and the .01 levels. 

Table 4.1 

Results of Seven 2x2x3x3 Analyses of Variance for 
Four Dependent Variables* 





Dependen 


Variables 


Source of Variation 




Discrimination 


Certaintv 


Consistencv 


Time 


Group- j 


Est. 


Rank- 


Est. 


Rank- 


Est. 








Cons. 


ing 


Cons. 


infi- 


Cons. 



1; 


Instruction 


.01 




2; 


Documents 


.05 


.05 


3: 


Masking 


.01 


.05 


4; 


Interview 


.01 


.01 


1 X 


2 


.05 


.05 



1x3 

1x4 

2x3 

2x4 

3x4 

1x2x3 

1x2x4 

1x3x4 

2x3x4 

1 X 2 X 3 X 4 



.01 .05 



.05 



.05 



.05 



* Table entries are maximum probabilities 



Time 



The analysis of variance, using time as a dependent variable , is 
shown in Table 4.2. 



1. The variables are explained in Chapter II and the procedures 
for their measurement and analysis are explained in Chapter III. 
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Table 4.2 

An Analysis of Variance Showing the Effect on Time 



Source of Variation 


1 d.fTT 


Sums of Squares 1 Mean Squares I 


F 




1. Instruction 


1 


11826.6 


11826.6 


8.975 


"kie 


2. Documents 


1 


5244.2 


5244.2 


3.98 


* 


3. Masking 


2 


13828.5 


6914.3 


5.25 


** 


4. Interview Infor- 


2 


116784.3 


58392.1 


44.3 


** 


matlon 












1x2 


1 


6601.6 


6601.6 


5.01 


* 


1x3 


2 


531.1 


265.6 


mm mm 




1x4 


2 


2273.4 


1136.7 






2x3 


2 


2681.7 


1340.8 


1.02 




2x4 


2 


2387.1 


1193.5 


— 




3x4 


4 


5852.3 


1463.1 


1.11 




1x2x3 


2 


1724.6 


862.3 


• • 




1x2x4 


2 


487.5 


243.8 


- <- 




1x3x4 


4 


775.0 


193.8 


<- 




2x3x4 


4 


1334.4 


333.6 






1 X 2 X 3 X 4 


4 


3239.5 


809.9 


-- 




Within Cells (Error) 


108 


142317.3 


1317.8 






Total 


143 


317888.9 









* p ^ .05 ** p ^ .01 



The following results are Indicated from the analysis: 

1. No Instruction required 18 minutes more time than Instruction. 

2. Multiple documents required 12 minutes more time than single 
documents* 

3. No masking required 271.5 minutes, idilch was significantly 
more than 257.8 minutes for partial masking, which was sig- 
nificantly more than 247.6 minutes for considerable masking. 2 

4. Audiovisual and audio Information were not significantly 
different (with mean times of 274.1 and 283.8 respectively), 
but both took a significantly longer time than no Inter- 
view Information (219.1 minutes). 

5. The effect of Instruction Interacted v;lth the effect of docu- 
ments In the following manner: When no Instructions were 

given, using the multiple documents required 25.6 minutes 
longer; however, when Instructions were given, there was no 
significant difference In the time required. This interac- 
tion is plotted In Figure 4.1. 



2. A Newman-Keuls test was used for all post analyses of means. 

it 
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Figure 4.1 




No Instruction Instruction 



Ihe interaction effect of Instruction (I) and Documents (D) 
on Time. 



Discrimination 



Results of the analysis of variance, using the grouping of appli- 
cants on various characteristics as a measure of discrimination are 
reported in Table 4.3 and indicate no differences for any of the depend- 
ent variables. (Two two-way interactions at the .10 level were not 
considered significant.) 

Using the estimate of ratings on the Teacher Evalution Instrument 
as a measure of discrimination for the ANOVA (see Table 4.4) yielded 
the following results: 

1. Instruction had no effect on this measure of discrimination. 

2. The single document produced more discrimination. 

3. No masking yielded more discriminating results than partial 
masking, which yielded more discriminating results than con- 
siderable masking. (Mean scores were 1.22, 1.07, and 0.96, 
respectively. ) 

4. The results of audiovisual and audio interview information were 
not significantly different, but the results of both were more 
discriminating than no interview information. (Mean scores 
were 1.15, 1.19, and 0.92, respectively.) 
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Table 4.3 

An Analysis of Variance Showing Effect on Discrimination 
as Measured by Number of Groupings 



Source of Variation I 


d.f. 1 


Sums of Squares | 


Mean Squares 1 


F 


1. Instruction 


1 


21.0 


21.0 




2. Documents 


1 


0.2 


0.2 




3. Masking 


2 


1.8 


0.9 


- - 


4. Interview Infor- 


2 


40.0 


20.0 


- - 


mation 










1x2 


1 


85.6 


85.6 


3 e 616 


1x3 


2 


37.5 


18.8 


-- 


1x4 


2 


143.0 


71.5 


3 e 022 


2x3 


2 


37.7 


18.8 


- - 


2x4 


2 


12.1 


6.0 


- - 


3x4 


4 


95.4 


23.9 


- - 


1x2x3 


2 


52.5 


26.3 


•• 


1x2x4 


2 


53.0 


26.5 


-- 


1x3x4 


4 


55.0 


13.8 




2x3x4 


4 


41.9 


10.5 


- - 


X 2 X 3 X 4 


4 


108.7 


27.2 


- - 


Within Cells (Error) 


108 


2555.3 


23.7 




Total 


143 


3340.7 







Table 4.4 

An Analysis of Variance Showing Effect on Discrimination, as Measured 
by Variance of Estimates of Consequences on 
the Teacher Evaluation Instrument 



Source of Variation I d.f. 1 Sums of Squares I Mean Squares I F 



1 0.09 

1 0,77 

2 1.55 

2 1.94 



1. Instruction 

2. Documents 

3. Masking 

4. Interview Infor- 
mation 

1x2 

1x3 

1x4 

2x3 

2x4 

3x4 

1x2x3 

1x2x4 

1x3x4 

2x3x4 

1 X 2 X 3 X 4 

Within Cells (Error) 

Total 



1 1.02 

2 0.25 

2 0.38 

2 0.05 

2 0.71 

4 0.98 

2 0.13 

2 0.17 

4 0.69 

4 1.05 

4 0.44 

108 20.53 

143 30.77 

* p ^.05 



0.09 


•• 




0.77 


4 e 06 


* 


0.78 


4.09 


* 


0.97 


5.10 


- k * 


1.02 


5.37 


* 


0.13 


-• 




0.19 


- - 




0.03 


- - 




0.36 


- - 




0.24 


- - 




0.07 


- - 




0.09 


- - 




0.17 


- - 




0.26 


- - 




0.11 


- - 




0.19 







** p < .01 
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5* The effect of instruction interacted with the effect of docu- 
ments in the following manner. VIhen no instructions were 
given > the multiple documents were less discriminating than 
the single documents. The Instruction appeared to depress the 
single documents' discrimination score fomewhat and increase 
the multiple documents' score considerably. Orthogonal com- 
parisons of the means indicated that the means that were dif- 
ferent were: (a) single and multiple documents when no instruc- 
tions were given (means: 1.219 and 0.905, respectively); and 

(b) the multiple documents when no instructions and Instructions 
were given (means: 0.905 and 1.124, respectively). This 

interaction is plotted in Figure 4.2. 



1 . 20 -- 



1.10 



Mean 

Discrimination 





■f 



+■ 



h 

No Instruction 



l2 

Instruction 



Multiple Documents 
Single Documents 



Figure 4.2 The interaction effect of Instruction (I) and Documents (D) 
on Discrimination, as measured by variance of estimates on 
the Teacher Evaluation Instrument. 



Certainty 

The analysis of variance regarding the certainty of the estimates 
of consequences on the Teacher Evaluation Instrument (see Table 4.5) 
yielded the following results: 

1. Hie effects of Instruction, documents, and masking were not 
significant on this measure of certainty. (The difference 
at the .10 level for instruction was not considered signi- 
ficant. ) 
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Table 4.5 

An Analysis of Variance Showing Effect on Certainty of Estimates of 
Consequences on the Teacher Evaluation Instrument 



Source of Variation 


1 d.f.n 


Sums of Squares 1 


Mean Squares { 


F 


1. Instruction 


1 


10.56 


10.56 


3e66 


2. Documents 


1 


2.51 


2.51 


-- 


3. Masking 


2 


2.26 


1.13 


— 


4. Interview Infor- 


2 


24.01 


12.01 


4.16 * 


mation 










1x2 


1 


0.34 


0.34 




1x3 


2 


8.38 


4.19 


— 


1x4 


2 


13.54 


6.77 


— 


2x3 


2 


2.51 


1.26 




2x4 


2 


23.35 


11.67 


4.04 * 


3x4 


4 


19.44 


4.86 


— 


1x2x3 


2 


3.18 


1.59 


— 


1x2x4 


2 


6.76 


3.38 


— 


1x3x4 


4 


16.08 


4.02 


— 


2x3x4 


4 


8.19 


2.05 


-- 


1 X 2 X 3 X 4 


4 


5.28 


1.32 


-- 


Within Cells (Error) 


108 


311.75 


2.89 




Total 


143 


458.16 







* p ^ .05 



2. There was significantly more certainty expressed with audio- 
visual than with audio Interview Information (mean scores: 

9.63» 8.65). No Interview Information yielded a mean score 
of 8.95, which was not significantly different from the other 
scores. 

3. The effect of the number of documents Interacted with the effect 

of the Interview Information treatment In the following manner. 
The audio depressed the certainty scores for the multiple docu- 
ment treatment below that of the single document, while the 
multiple document treatment exhibited more certainty with the 
audiovisual and no Interview Information. This Interaction is 
plotted In Figure 4.3. Two comparisons were made of the Inter- 
action effects: (a) between levels A1 and A2, and (b) between 

levels A2 and A3. Since these were not orthogonal comparisons, 
Scheffe's test of multiple comparisons was used and the level 

of significance was set at .10, suggested by Scheffe'^ because of 
die conservatism of the test, rather than the more normal level 
of .05. The F*s for the two comparisons were 6.06 and 6.07, 
respectively, well above the required F* of 4.72 for significance. 



* 



ir 



10.0 



9.0 ” 



Mean 

Certainty 



8.0 




l^ltlple 

Documents 



Single 

Documents 




Al A2 A3 

No Interview Audio A-V Interview 

Information Information Information 



Figure 4.3 The Interactlc i effect of Interview Information (A) and 
Documents (D) on Certainty of estimates on the Teacher 
Evaluation Instrument. 



The analysis of variance regarding the certainty of the ranking of 
applicants for the position (see Table 4.6) yielded the following results: 

1. The effect of Instruction, documents, and masking was not signi- 
ficant on this measure of certainty. 

2. Audiovisual Interview Information yielded significantly more 

certainty than either audio Information or no Interview Infor- 
mation. (Mean scores: audiovisual, 9.92; audio, 8.50; no 

Information, 9.04.) 

3. The effects of the four variables Interacted with regard to 
this measure of certainty. 

Consistency 

The analysis of variance, using as a measure of consistency the rank 
order correlations between event one and event two (morning and after- 
noon sessions) for the ranking of candidates for the position, yielded 
no differences for any of the Independent variables. (One three-way 
Interaction at the .10 level was not considered significant.) The data 
are reported In Table 4.7. The variance within groups was so great on 
the consistency of rank order of the candidates from morning to after- 
noon session that differences did not appear. The small number of 
candidates (n - 5) used In the rank order correlation reduced the 
possibility of obtaining significant differences; likewise, this may 
partially account for the large variances In correlation coefficients 
within cells. However, similar wlthln-cell variances were also noted 
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for correlation coefficients between the two estimates of consequences 
on the 49-item Teacher Evaluation Instrument. The reason for such 
high cell variance appears to be related to the heterogeneous back- 
grounds of the subjects Involved, but further investigation of this 
phenomenon is warranted. 



Table 4.6 

An Analysis of Variance Showing Effect on Certainty 

of Ranking of Applicants 



Source of Variation 1 d.f. 


1 Sums of Squares 


1 Mean Squares 


1 F 


1. Instruction 


1 


1.36 


1.36 




2. Documents 


1 


0.00 


0.00 




3. Masking 


2 


2.26 


1.13 


-- 


4. Interview Infor- 


2 


49.06 


24.53 


6.02 ** 


matlon 










1x2 


1 


2.78 


2.78 




1x3 


2 


5.01 


2.51 




1x4 


2 


14.39 


7.19 




2x3 


2 


2.63 


1.31 




2x4 


2 


7.17 


3.5 




3x4 


4 


19.69 


4.92 




1x2x3 


2 


3.93 


1.97 




1x2x4 


2 


5.72 


2.86 




1x3x4 


4 


31.86 


7.97 




2x3x4 


4 


18.33 


4.58 




1 X 2 X 3 X 4 


4 


52.44 


13.11 


3.218 * 


Within Cells (Error) 


108 


440.00 


4.07 




Total 


143 


656.64 








* P 


05 ** P S 


.01 








Table 4.7 






An Analysis 


of Variance Showing Effect 


on Consistency 






of Ranking of Applicants 




1. Instruction 


1 


0.00 


0.00 




2. Documents 


1 


0.02 


0.02 




3. Masking 


2 


0.88 


0.44 




4. Interview Infor- 


2 


0.98 


0.49 




mation 










1x2 


1 


0.07 


0.07 




1x3 


2 


0.22 


0.11 




1x4 


2 


0.48 


0.24 




2x3 


2 


0.13 


0.07 




2x4 


2 


0.85 


0.42 




3x4 


4 


0.08 


0.02 




1x2x3 


2 


1.92 


0.96 




lx 2x4 


2 


0.36 


0.18 




1x3x4 


4 


2.23 


0.56 




2x3x4 


4 


0.58 


0.14 




1 X 2 X 3 X 4 


4 


0.53 


0.13 




Within Cells (Error) 


108 


38.78 


0.36 


* 


Total 


143 


48.10 
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The analysis of variance, using as a measure of consistency the 
correlation of the estimates of consequences on the Teacher Evaluation 
Instrument of event one with event two (see Table 4.8) yielded the fol- 
lowing results: 

1. There were no main effects of the Independent variables for 
this measure of consistency. 

2. The degree of masking Interacted with the document variable, 

and the Interaction Is plotted In Figure 4.4. Observation of 
the figure Indicates that the Intereactlon was 'caused by the 
Increase In consistency from M2 to M3 with the multiple docu- 
ments, and the concomitant decrease In consistency with the 
single document. An additional analysis Indicated that this 
Interpretation Is correct. An analysis was made of two com- 
ponents of the Interaction: (a) the combination of levels Ml 

and M2 compared with the M3 level (labeled ml x D); and (b) 

a comparison of level** Ml and M2 (labeled m2 x D) . ml and m2 
are a set of orthogonal comparisons as shown by the following 
weights : 

Ml M2 M3 
ml 11-2 

m2 1-10 

Table 4.9 shows the components of the Interaction. 

Table 4.8 

An Analysis of Variance Showing Effect on Consistency 
of Estimating Consequences on the Teacher Evaluation Instrument 



Source of Variation 1 


d.f. 


1 Sums of Squares 1 Mean Square 1 


F 


1. Instruction 


1 


0.05 


0.05 


Mi 


2. Documents 


1 


0.03 


0.03 




3. Masking 


2 


0.11 


0.05 


mm « 


4. Interview Infor- 


2 


0.11 


0.05 




mation 










1x2 


1 


0.01 


0.01 


. - 


1x3 


2 


0.07 


0.04 


mm mm 


1x4 


2 


0.18 


0.09 


H di 


2x3 


2 


0.50 


0.25 


4.18 * 


2x4 


2 


0.02 


0.01 


- - 


3x4 


4 


0.45 


0.11 




1x2x3 


2 


0.18 


0.09 




1x2x4 


2 


0.11 


0.05 


-- 


1x3x4 


4 


0.23 


0.06 


mm M 


2x3x4 


4 


0.36 


0.09 




1 X 2 X 3 X 4 


4 


0.07 


0.02 




Within Cells (Error) 


108 


6.41 


0.06 


M mm 


Total 


143 


8.87 








Mean 

Consistency 
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1 


Ml 
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Figure 4.4 The Interaction effect of Masking (M) and Documents (D) on 

Consistency of estimates on the Teacher Evaluation Instrument. 



Table 4.9 

An Analysis of the Components 
of the Documents by Masking Interaction 



Source of Variation I "d.f. I Sums of Souares^l Mean Squares I F 



2x3 (Documents by 
Masking) 


2 


0.50 


0.25 


4.18 * 


ml X D 


1 


0.496 


0.496 


8.52 ** 


m2 X D 


1 


0.004 


0.004 


• mm 


(Error) 


108 


38.78 


0.36 





* P ^«05 ** p <.01 



Two cautions are Interjected at this point regarding the results of the 
analysis of the consistency measure. First, It will be noted In Table 4.8 
that 15 F tests were computed, and only one of them ~ a two-way Inter- 
action —* was significant at the .05 level. It Is quite likely that this 
Is a chance event. In that none of the main effects were significant and 
only one of the 15 tests was significant. If all of the degrees of 
freedom of the sources of variation had been the same. It would have been 
possible to get an estimate of the likelihood that this was a chance 
event by comparing the mean square of the significant Interaction with 
the largest of the other mean squares and treating the ratio as an F 
score. Since the degrees of freedom were not equal, this comparison 
was not possible. In spite of the warning, the reader Is reminded that 
the results are Interpretable within the realm of the Idea that large 
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amounts of information are unmanageable and small amounts lead to incon- 
sistency* In addition, the eventual recommendations would not be dif- 
ferent even if this interaction is due to chance* 

Discussion of the Results 

The presentation of the results in terms of the effects on the four 
dependent variables provides a certain perspective regarding the outcomes 
of the experiment* However, another perspective can be obtained by 
analyzing what effect each of the Independent variables had* This 
latter provides more insight into the implications of the study and the 
recommendations to be made as a result of the study* 

Instruction 



Instruction on how to process information, under these experimental 
conditions, reduced the amount of time it took to make decisions* 
Instruction also Interacted with documents by reducing the time for 
multiple documents and by increasing the discrimination of the subjects' 
estimates on the Teacher Evaluation Instrument* 

Therefore, regardless of the format of the information, there would 
be a time benefit from giving information -processing instructions to 
principals who are engaged in the selection of teachers* However, 
Increased discrimination would result from instructions only where 
multiple documents were used (which happens to be the more normal pro- 
cedure at present, although the advantages of the single document are 
indicated in the following section)* 

Documents 



The single document reduced the time it took to make decisions and 
increased the amount of discrimination in making estimates on the Teacher 
Evaluation Instrument* However, documents interacted with interview 
information by depressing the certainty of estimate on the Teacher 
Evaluation Instrument for the audio Interview Information obtained by 
the multiple document treatment* 

Documents also Interacted with the masking information as far as 
consistency was concerned in estimation on the Teacher Evaluation 
Instrument* (As Indicated earlier, this interaction may be due to 
chance, since no main effects or other interactions were significant 
for the 15 F tests computed for this ANOVA*) When multiple documents 
were used the most consistency was obtained with considerable masking; 
when the single document was used, it yielded the most consistency with 
no masking* These results appear to be compatible with a general notion 
that too much information (or information in an unmanageable form) is 
confusing and precipitates Inconsistent responses, while too little 
information precipitates random behavior* 

The main effects of the document variable indicate the advantages 
of the single document for savings of time and for Increasing the dis- 
crimination of decision makers. The interaction effect with the inter- 
view information is difficult to explain; but since it is primarily 



concerned with the relationship of the multiple documents with audio 
Information (a case which is not commonly used and is not recommended 
as part of the optimum -in format ion format), this should not cause undue 
concern. The interaction between documents and masking, although quite 
possibly due to chance, points to a potential advantage of the single 
document (or of masking, if multiple documents were used) as far as 
consistency was concerned. 

Masking 

The degree of masking of information had a main effect of reducing 
the time needed to make decisions and by decreasing the discriminations 
made in the estimates of consequences on the Teacher Evaluation Instru- 
ment. Also, as indicated in the prior section, masking reduced the 
consistency of single documents. 

An Increased amount of masking provided for an advantage of saving 
of time; however, the Increased masking reduced the discrimination. 

Of the two of these effects, the discrimination is probably more important 
as far as risk and potential loss is concerned. As indicated above, the 
documents -masking interaction points to the advantages of either the 
single document with no masking or of the multiple documents with mask- 
ing of non -exceptional data. 

Interview Information 



The analysis of the effect of the interview information indicated 
that audiovisual and audio information Increased the time needed to make 
decisions, but increased the discrimination on the estimation of conse- 
quences on the Teacher Evaluation Instrument. In neither of these cases 
was there a significant difference between the results of the audiovisual 
and the audio treatments. 

The results also Indicate that there was more certainty expressed 
with the audiovisual information than with the audio on both measures 
of certainty, but only with the measure of certainty regarding the rank- 
ing of applicants was the mean certainty of the audiovisual greater 
than with no interview information. The interaction of interview infor- 
mation with documents (where certainty was measured by the estimation 
on the TEI) indicated that the audio information depressed the certainty 
scores with the multiple documents. 

The main effects of the interview variable Indicate there are ad- 
vantages to having audiovisual information as far as discrimination and 
feelings of certainty regarding decision making are concerned; however, 
there is a concomitant loss in time. The advantages of the main effects 
appear to outweigh the disadvantage of loss in time. The comparisons of 
the three types of information indicate that the only advantages of the 
audiovisual interview information over the audio interview information 
(on this set of dependent variables) is in the relatively higher feelings 
of certainty. However, the interview-documents interaction indicates a 
relative disadvantage to audio interview information presented in combina- 
tion with multiple documents. 
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Summary 



The results of the study were presented with relation to the effects 
of the dependent variables and indicate: 

!• Time to process information and make decisions regarding teacher 
applicants was: (a) reduced by instruction, (b) reduced by 

the single document, (c) reduced by masking, and (d) increased by 
audio or audiovisual interview information. In addition, inter- 
action between instruction and documents indicated the instruc- 
tion reduced the time of subjects who had multiple documents 
more than those with single documents. 

Discrimination was: (a) increased by the single document, 

(b) decreased by masking, (c) increased by audio and audio- 
visual interview information, and (d) not affected by the 
instruction variables. In addition, the instruction-document 
interaction indicated the discrimination was increased when 
the multiple documents treatment was combined with instructions 
regarding how to process information, 

3, Feelings of certainty regarding decisions were: (a) increased 

by the use of audiovisual interview information, and (b) not 
affected by the instruction, documents, and masking variables. 

In addition, the document -interview interaction (for the certainty 
regarding the estimates of consequences on the Teacher Evaluation 
Instrument) indicated the audio information depressed the cer- 
tainty scores for the multiple documents treatment, 

4, Regarding consistency of decision making, there were no main 

effects of instruction, documents, masking, or interview infor- 
mation, Using the correlation of the estimates of consequences 
on the Teacher Evaluation Instrument of event one with event 
two as a measure of consistency, there was one Interaction 
between documents and masking — indicating more consistency 
with: (a) single documents and no masking, and (b) multiple 

documents and considerable masking. 

The discussion of these results, according to the effect that each 
independent variable had, indicated: 

Instructions on how to process information reduced the amount 
of time it took to make decisions. In addition, instructions 
had more effect on multiple documents than on single documents. 

The single documents reduced the time needed to make decisions 
and increased the discrimination in making estimates on the 
Teacher Evalution Instrument, In addition, the single docu- 
ment with no masking or partial masking was more consistent 
than with considerable masking. The multiple documents combined 
with the audio interview information reduced the feelings of 
certainty regarding estimates on the Teacher Evaluation Instru- 
ment, 



3. The masking of Information ; (a) reduced the time needed to 
make decisions* and (b) decreased the discriminations of esti- 
mates of consequences on the Teacher Evaluation Instrument* 

In addition, as noted above, masking Interacted with documents. 

4* The audiovisual Interview Information presented to subjects: 

(a) Increased the time needed to make decisions, (b) Increased 
the discrimination on estimates of consequences on the Teacher 
Evaluation Instrument, and (c) Increased both measures of 
certainty. There appeared to be little difference between the 
effect of audio and audiovisual Interview Information on the 
time and discrimination measures. 

The results of this study were not concerned with the validity of 
the decisions. It was assumed that the Importance of situational 
variables necessitates local validation of selection decisions and that 
this local validation should be done with an optimum Information format 
as far as time, certainty, discrimination, and consistency are concerned. 
Additional study In the simulated situation should allow us to determine 
whether subjects will be able to make valid decisions for predetermined 
and specified criteria with the Information format recomnended. 
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CHAPTER V, CONCLUSIONS, IMPLICATIONS, AND RECOiiMENDATIONS 

Prior chapters have presented the background and purposes of the 
study, the procedures used, and the results and findings. This chapter 
will present the major conclusions, implications for practice and future 
research, and some recommendations based on these conclusions and impli- 
cations. 



Conclusions 

The major conclusions of the study were: 

1. The use of a simulated situation facilitated the study of the 

teacher selection decision process by allowing variables to be 
manipulated — and the results of this manipulation to be ana- 
lyzed. The simulated situation included: (a) the description 

of the hypothetical community; (b) the presentation of fictitious 
applicants via written documents, such as credentials and appli- 
cation forms, and filmed interviews; and (c) a set of decisions 
to be made regarding the applicants. 

2. The format of information provided to the subjects of the exper- 
iment did have an effect on the decisions they made. Each of 
the dependent variables was affected by at least one of the 
manipulated variables. 

3. The optimum format for minimizing time and maximizing discrimin- 

ation, certainty, and consistency under the experimental condi- 
tions was the combination of: (a) instructions reeardine the 

processing of information, (b) a single summary document , (c) 
n£ masking of information on the single document, and (d) inter - 
views that include visual as well as audio stimuli. 

4. The differences between using audiovisual and audio interview 
information were in the subjects' feelings of certainty regarding 
judgments made; otherwise, comparable results were obtained 

from these two forms of interview information. 

Implications 

The implications of this research project for practice in the selec- 
tion of teachers are as follows: If principals or personnel directors 

involved in selection of teachers are of a similar nature to the subjects 
used in this study, their decisions regarding the selection of teachers 
will be affected by the format of Information about applicants. Further, 
one would expect that the format which would yield optimum results (as 
far as time, discrimination, consistency, and certainty are concerned) 
would consist of the format found optimum in this study, viz . . instructions 
regarding the processing of information, a single summary document , no mask - 
ing of information, and interviews that include visual as well as audio stimuli. 

The implications of this research project, resulting from the mater- 
ials developed, are considerable in relation to increasing the knowledge 
of decision-making processes. An optimum information format will allow 
experiments to be conducted in a simulated situation without fear that the 



- 45 - 



results will be adversely affected by the manner of presenting information 
to subjects. This experiment, then, was necessary in order to utilize the 
simulated situation for testing hypotheses regarding certain elements of 
decision theory. For example, if one wants to describe the decision- 
iQuldng behavior of a particular individual (or a set of individuals), or 
if one desires to prescribe a manner in which a decision maker might 
behave more effectively, it is necessary to determine both; (a) the 
manner in which he predicts consequences, or at least what consequences 
he predicts, and (b) the value system he uses in the final choice. But 
how can the prediction of consequences be separated from the values 
attached to them? How does one know, by observing the choice of a par- 
ticular alternative, whether the choice was made on the basis of a high 
prediction of consequences and a low value, or the reverse, or both a 
high prediction of consequence and a high value? 

Interest in value systems has led students of decision making to 
devise descriptive and prescriptive decision-making models. One intent 
of these models is to assist people in making the consequences they pre- 
dict and the values attached to them explicit, yet little wyrk has been 
done to accomplish this Intent. 

One approach to the description of the decision-making process might 
be to place subjects in a precisely described choice situation in which 
the consequences can be accurately determined and known by the subject. 

For example, a betting situation in which the odds were known—as in 
coin-flipping, rolling dice, or choosing combinations from a deck of 
cards— might be used. Subjects could be taught the probabilities of 
certain consequences occurring and their values could be inferred from 
the alternatives chosen, l«e., the types of bets they made. However, 
such an approach would leave much to be desired, because prediction of 
consequences would have been controlled, in a sense; therefore, one 
could only infer that differences in behavior were due to differences in 
value systems rather than the way consequences were predicted. Inability 
to determine concomitantly the subject's manner of predicting consequences 
and his value system is a limitation in this situation; such a limitation 
might elicit behavior considerably different from behavior in a less 
restricted decision situation. 

An approach from which broader generalizations might be made is one 
in which the situation is described, but the decision maker must make 
choices on the basis of his own prediction of consequences and attachment 
of values to these consequences. An example is the simulated teacher 
selection situation described in this report. In this decision situa- 
tion, subjects are not taught probabilities of consequences of choosing 
certain teacher applicants but must make estimates of what will occur 
if each teacher is hired. In addition, they must make choices among 
the teachers. The estimates of what will occur When a teacher is hired 
become the subject's explicit expression of probable consequences, and 
the value system of the subject is implied by this expression and his 
choices among teachers. 

The materials used to simulate the decision-making process for 
selecting teachers, then, provide a setting whereby descriptive and 
prescriptive theories of decision making may be tested. In addition. 
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other variables that have been controlled in this experiment— especially 
those dealing with situational factors and interview information— can be 
manipulated in future experiments to determine their contribution to 
decision making* The results of this project, in addition to providing 
some recommendations for the practices of selecting teachers, make pos- 
sible the control of a very important variable ( viz . , the format of 
information) in future studies of the decision-making process. 

Recommendations 

The recommendations to be made as a result of the experiment are: 

1. Administrators of school district should: 

a. Develop ways to use the format which was found to be optimum 
in this study in providing information to decision makers to 
select teachers. The potential saving in time and increase 
in discrimination, certainty, and consistency are too great 
for school administrators not to adapt the results of this 
study to their particular needs. 

b. Not hesitate to use audio-interview information where it 
is necessary or expedient (for example, telephone inter- 
views with persons who are considerable distances from 
the location of employment). 

2. Programs should be devised on a state or national basis to assist 
school districts to: 

a. Develop single, summary documents appropriate for local use. 
Dissemination of the results of this study will be sufficient 
for some school districts to modify their practices. However, 
it would be beneficial to many districts if a developmental 
activity were engaged in whereby the dimensions of the infor- 
mation format and its relation to local peculiarities were 
fully explored. For example, various types of single summary 
documents might be developed and field tested in order that 
their advantages and disadvantages could be known to local 
school districts. 

b. Provide instructional programs regarding the processing of 
information for selection of teachers. The materials developed 
for this project provide considerable potential for instructing 
people regarding the selection of teachers — under circumstances 
whereby feedback of results can be provided. However, since 
the materials were developed for purposes of conducting an 
experiment, they should be adapted for instructional purposes. 
In addition, the results of the instructions on how to process 
information were encouraging enough to warrant further devel- 
opment and utilization. 

3. Additional research and development activities should be initiated 
and corducted to determine: 
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a. The conditions under which individuals can learn to select 
teachers for specific selection criteria representing the 
value system of a specified school district. Although this 
project provides an optimum format for presenting informa- 
tion to decision makers for selecting teachers t the format 
does not provide the requirements for valid decisions. Valid 
decisions can be made only for specified criteria, and it 
has been assumed that individuals can learn to make valid 
decisions if the information format does not interfere with 
the learning process. However, this assumption should be 
tested and conditions should be identified whereby individuals 
can learn to select teachers for specified criteria. The 
materials developed by this project should be useful in such 

a developmental program, but they should be field tested for 
their instructional utility. 

b. How differing value systems of individuals interact with the 
abilities of the individuals to estimate consequences of 
alternatives. This research project did not acquire measures 
of the value systems of subjects (either as implied by their 
decisions or as explicity stated by the subjects), since 

the concern was to develop an information format that would 
not adversely affect the decision process. Because an im- 
portant facet of decision making is the interaction of value 
systems with the estimation of consequences, this should be 
studied in the simulated situation— using the optimum format 
for presenting the information. 

c. How certain variables controlled in this experiment (e.g., 
grade level of the vacancy, sex of the applicants, age range 
of the applicants, racial and ethnic background of appli- 
cants) affect the decision making of administrators. It is 
often said that the selection of teachers depends on the 
situation for which the teacher is being selected, but no 
empirical evidence is available to indicate that administra- 
tors actually make adjustments in their selection processes 
for situational variables. If they do not, they tend to 
select for some stereotype of a global "good teacher," where- 
as the research evidence is plentiful that no uniformly good 
teacher exists. 

d. The effect of the instructional variable on other populations 
and with varying amounts and types of instruction. For 
example, will superintendents and directors of personnel 
profit from instruction regarding hc.7 to process information? 
Likewise, will more extended instruction sessions that in- 
clude discussion of situational variables, and their relation 
to the criteria to be used, be beneficial to decision makers? 
Will the same effect occur regardless of the experiential 
background of the subject? Will it be necessary to give 
instructions each time the task is performed? Will the in- 
structional effects transfer to other information-processing 
tasks? All of these questions warrant further study. 
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e. Why the interaction between degree of masking and single 
documents resulted in reduced consistency. Also, there is 

a need to know why masking reduced discrimination. Are these 
results due to the fact that not enough information is avail- 
able to make discriminating and consistent decisions, or is 
it due to the lack of familiarity of principals with decision 
processes that emphasize attending to exceptional data? 
Studies need to be completed that will determine whether 
additional familiarity with exceptional-information-decision- 
making procedures yields different results. If so, time 
might be saved by using some procedure of partial masking. 

f. Whether the lack of difference between the audio-interview 
information and the audiovisual -interview information is con- 
sistent under other ciucumstances. For example, if instruc- 
tion in specific techniques of observation and listening 
skills are provided, will differential results occur? The 
use of televised or live interviews could facilitate the 
teaching of observation skills, and the materials developed 
for this project should allow the testing of hypotheses 
regarding the effect of such instruction on decision making. 

g. Why the nature of the interview information affects differ- 
entially the certainty of decisions made with different types 
of documents. If the certainty measured with the "no inter- 
view information, multiple documents" had been low in the 
same manner as the "audio information, multiple documents," 

a rather simple explanation is available. However, reduc- 
tion of certainty with audio information and the lack 
of reduction of certainty with no information is puzzling 
and needs further investigation. 

h. How data processing equipment could be used for reliably 
transferring data from multiple documents to single docu- 
ments. For this study, a strategy was developed whereby 
these data were transferred, but research needs to be con- 
ducted that will allow data processing equipment to be 
used for a variety of strategies. 

Summary 

This experiment was conducted in a simulated situation which facili- 
tated the study of the teacher selection decision process by allowing 
information-format variables to be manipulated. It was concluded that 
the format of information provided to the subjects of the experiment 
(elementary school principals) did have an effect on the decisions made 
and that the optimum format consisted of instructions regarding the proces- 
sing of information, a single summary document, no masking of information, 
and audiovisual interviews. 

The implication of the study for the practice of selecting teachers 
is straightforward: Improvements could be made in the selection of 

teachers by using the format found to be optimum in this study. The 
improvements expected would be decreased time spent in making decisions, 
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and increased discrimination, certainty, and consistency regarding the 
decisions made. 

The materials used to simulate the decision-making process for 
selecting teachers provide a setting whereby descriptive and prescrip- 
tive theories of decision making may be tested. The results of the pro- 
ject make it possible to control a very important variable, the format 
of information, in future studies of the decision-making process. 

It is recommended that additional studies be conducted to determine 
the effect of other variables on the teacher selection decision process. 
In addition, it is recommended that school administrators, and state 
and national governmental agencies engage in developmental activities 
that would implement the utilization of the optimum information format 
found in this study. 
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APPENDIX A 

The Teacher Selection Process^ 



If the classification system specified by Cronbach and Gleser 
(1965:15-17) is used to describe the decision process for selecting 
teachers, the following Is Indicated: 

1. The person selecting teachers is Involved in making an insti- 
tutional decision rather than an individual decision. An 
institutional decision is one where a common set or values is 
applied to a large number of comparable decisions. There Is 

an attempt to make decisions In such a manner that the institu- 
tion benefits as much as possible. The applicant, however, 
must make an individual decision to join the organization, 
and this is made on the basis of individual values. 

2. Each teacher applicant is assigned to a single treatment. A 
teacher cannot be assigned to a "hire" treatment and a "not 
hire" treatment at the same time; neither can a teacher be 
assigned to a fourth-grade self-contained classroom and also 
be assigned to teach art at a junior high school. At times 
the assignment Is adapted to a teacher by modifying the spe- 
cifics of the assignment to the peculiar abilities of the tea- 
cher. For example, a teacher may be hired for a fourth -grade 
assignment; but rooms, materials, or students may be changed 
because of the abilities of the teacher. 

3. A quota is rather precisely followed. A school district has 

a number of vacancies to fill and only that number of teachers 
is hired. At times a district may hire a few extra elementary 
teachers with the anticipation that late resignations will 
occur, but seldom Is this the case. 

4. In all decisions, one of the acceptable treatments is "reject." 
This assumes that the selection ratio, i.e., the ratio of the 
number selected to the number who applied, is less than 1.0. 

If a district is In such an unfavorable geographical location 
that very few applications are received, the selection ratio 
might Indeed be 1.0. In this case, no rejections would occur. 
On the other hand, it is inconceivable that any district would 
have a policy where "reject" Is not a possibility. Other per- 
sonnel decisions are made under conditions where "reject" is 
not a possibility (for example, admission of normal students 
to public schools). 



1. This description of the Teacher -Selection Process appeared as 
part of a paper read by Dale L. Bolton at the 1967 American Educational 
Research Association meeting in New York City, entitled "Feedback in a 
Selection of Teacher Simulation." 
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5. Generally, the information gathered is in a multivariate form. 
Seldom is grade point average, or rating on personality in an 
interview, or recommendations regarding reliability considered 
separately. The total information gathered represents a variety 
of dimensions; it may be factor analyzed to reduce its complex- 
ity, but it still represents multiple variables. 

6. The decision is commonly made at any one of various points in 

a sequence of information gathering, i.e., a type of sequentifei 
testing or successive hurdles is used. If cutting points are 
used with any of the information categories, it Is not even 
necessary that the information be processed in a prescribed 
sequence. However, all information may be collected and then 
a single final decision made. 

If it is assumed that all information about the teacher apnlicants 
is collected and then a single final selection decision is made*, the 
data might take the form of multivariate information collected to pri^:- 
dict various behaviors which have varying utility in relation to some 
institutional goal. 

The tasks of the person who selects teachers include: (a) collect- 

ing information in such a manner that it is reliable, (b) using this 
information for predicting the consequent behaviors of the teacher, and 
(c) relating all of these behaviors to the operation of the organization 
so that some measure of the total utility of the individual to the organ- 
ization may be made. 

These tasks are necessary to determine the relative merit of each 
applicant for a specific assignment. In addition, of course, the deci- 
sion maker must determine how many — if any — of the applicants should be 
hired at a particular time. This decision depends on the quota to be 
filled at the time, the quality of the applicants being considered, the 
probability that additional persons will apply, and the probable quality 
of such additional applicants. The number of additional applicants and 
their quality are related to the time of the year. 

We may ignore temporarily the problem of deciding whether or not 
to hire and concentrate on rank ordering the available applicants on 
the basis of some over -all contribution to the major goals of the organi- 
zation. 



Let us assume that we can collect information regarding a group of 
teachers who are already members of a school system and that we are able 
to collect information regarding applicants for this school system. It 
could be arranged in tabular form as in Table A-1.^ 



2. This is the more general case (in the sense of encompassing the 

sequential testing case), and any investigatory decision is a special 
case with the availability of an additional treatment: collect additional 

information of a specific nature. 

3. See Horst (1962:253) for an explanation of relationships between 
predictor and criterion attributes that use a four-cell model. 



The Information Categories (Y) may represent biographical entries 
on an application blank or credential form, test scores, grades, ratings 
by people who have knowledge of specific or over-all behavior of the 
Individual, or Interview ratings. The Consequent Behaviors (C) are the 
behaviors exhibited by teachers In job-related activities. Examples of 
such behavior might Include warmth and friendliness or verbal facility 
exhibited. These behaviors may be specified by a given district and may 
be reliably observed by trained observers. They may take the form of 
ratings on factors such as those Identified by Ryans In his Teacher 
Characteristics Study (1960;388-93). The Total Utility (U) Is a measure 
of the value of the Individual to the operation of the system. This 
measure may be In the form of a scaled value, a rank ordering, or a 
clustering of people Into groupings or categories. 

Table A-1 

Relation of Information, Consequent Behavior and 
Total Utility In the Selection of Teachers 





Person 


Information 
Categories (Y) 


Consequent 
Behaviors (C) 


Total 

Utility 


1 2 .... m 


1 2 • • • • o 


U 


Members 
of the 
School 
System 


1 

2 

• 

• 

• 

N 


I 


II 


III 


Applicants 


N + 1 
N + 2 

• 

• 

• 


IV 


V 


VI 



It should be emphasized at this point that It Is assumed that the 
consequent behaviors and the utility are sltuatlonallv determined; no 
combination of Y's will yield C*s of a particular nature regardless of 
situational factors; the value or utility of a given set of C's Is 
determined In a situational context, rather than being uniform across 
situations. For example, a teacher may exhibit much more verbal facility 
with a group of senior honors students then with an average group of 
sophomores. Likewise, a rural district may value verbal facility differ- 
ently from a suburban district. 

The task, then, as formulated. Is to decide which applicants are 
more likely to make the most valuable over-all contribution to the major 
goals of the organization. This decision Is made by predicting conse- 
quent behaviors (C) and by attaching values to these behaviors. The 
prediction of consequent behaviors Is made on the basis of Information 
collected (Y), and the attachment of values Is based on Institutional 
goals. 
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Some of the elements which must be considered in teaching a person 
to make such decisions (l.e., which of several teachers to select) 
include the following! (a) knowledge of relationships between various 
Y and C categories in the present members of the system (or in the 
treatment category for which selection is to be made); (b) knowledge of 
the relationships between various C categories and U in the present mem- 
bers of the system; (c) practice in using the relational knowledge on 
applicant groups to predict |C and to determine U; (d) feedback regarding 
the effectiveness of the decisions made. 



APPENDIX B 

Presentation of Information Regarding the Situation 
via Slides and Recorded Narration 

Section One. Location and Physical Setting of the Community 

This section presented information regarding location and geographic 
features of the hypothetical community of Norwest in the State of Columbia. 
Two slides were shown of maps of the hypothetical state and county. A 
sequence of six slides illustrated via maps the various district of 
Norwest (e.g., business district, new and old industrial districts, and 
new and old residential districts) and the narration indicated how each 
district had developed. 

Section Two. Nature of the Community 

This section developed concepts of the community regarding its his- 
tory, growth, leadership characteristics, religious and recreational acti- 
vities, and health and safety facilities. There were 32 slides in this 
section, and approximately half of them illustrated the community's 
industrial and economic activities. The remainder of the slides illus- 
trated the business district, new shopping centers, the newly-developed 
cultural center, police and fire departments, churches, recreational 
facilities, and a hospital. The narration explained the services and 
civic activities connected with the slides and discussed the general 
growth of the social and economic aspects of the community. 

Section Three. The School District 

This section presented a brief history of the present consolidated 
school district, its growth characteristics, and future needs. Also 
included in the section was information regarding the characteristics 
of the school board, the organizational structure of the school system, 
and a general description of the school programs at the secondary and ele- 
mentary levels. Ten slides were used to illustrate the administrative 
offices, the organizational structure and various schools in the district. 

Section Four. Norwood Elementary School 

The attendance area was described with five accompanying slides of 
the homes in the area, l^ile the subjects were viewing twelve slides of 
interior and exterior scenes of the school, the narration presented de- 
tailed information regarding the teaching staff, the principal, and the 
instructional program of the school. 

Section Five. The Vacancy 

Seven slides presented views of the classroom to be used by the 
teacher to be selected. The narration specifically focused on a fourth- 
grade vacancy, e.g., pupil -grouping characteristics, supportive personnel, 
the fourth-grade instructional program, and the other fourth-grade 
teachers with whom the selected teacher V70uld be working. 
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APPENDIX C 

Nature of Programed Text 



A programed text, utilizing the branching or scrambled-book tech- 
nique, was developed to present essentially the same content regarding 
the situation as the audio-taped comn«ntary. A frame of information with 
a question and alternative answers was presented on a page in the book. 
Subjects in the experiment were directed to different pages depending 
upon their choice of an answer. In case of an incorrect response, the 
subject was either directed to return to the frame to select another answer 
or given supplemental information in order to choose the correct answer. 

The correct response was reinforced as being correct and then followed 
by the next frame of information. 

The four sections of the programmed text and their content were! 

Section One - Location and Physical Setting (3 frames). 

This section included: (a) the location of the city within 

the state; and (b) areas of the city, especially the areas 
of residential and industrial growth. 

Section Two - Nature of the Community (8 frames). 

This section included: (a) present industrial and community 

activities; (b) leadership structure in the community; and 
(c) potential of the community. 

Section Three - Norwest Community School District (6 frames). 

This section included: (a) organizational structure of the 

district; and (b) district growth and expectations. 

Section Four - Norwood Elementary School (15 frames). 

This section included: (a) characteristics of the teaching 

staff and principal; (b) instructional program; (c) conversation 
of principal and assistant superintendent of personnel regard- 
ing supply of applicants, expected salary range; and (d) excerpts 
from the job description. 



APPENDIX D 

Development of the Fictitious Applicants 

The problem confronted in designing the simulated interview situa- 
tions was to create "personalities" for the 16 fictitious applicants 
for the teaching position. In essence, eleven distinct types were re- 
quired, since five of the applicants appeared in both the morning and 
afternoon sets of applicants^ It was necessary for these five applicants 
to have equivalent characteristics in both the morning and afternoon ses- 
sion in order to obtain an accurate reliability measure. For all eleven 
applicants, it was desired to create "personalities" that were clearly 
distinct from one another, and yet were within the range of homogeneity 
established as a control factor in the experiment. If the applicants 
were too diverse, few differences would be likely to appear among the 
various treatments. 



After reviewing the literature relevant to teacher personality and 
behavior, it was decided to utilize the work of David Ryans in the 
Teacher Characteristic Study (Ryans, 1960) because this was considered 
to be the most thorough and precise study in the area of teacher behavior 
variables. Inasmuch as the setting for the experimental task was an 
elementary school, use of the five factors developed in the Teacher 
Characteristics Study for the elementary teacher sample was considered 
appropriate. These five factors were: originality, systematic, empathy, 

sociability, and buoyancy. 

To provide a framework on which to develop the ficititious appli- 
cants, these five factors were systematically manipulated so that each 
applicant would be high on one factor, two other factors would be pre- 
sent but not predominant, and the remaining two factors would be neutral 
or not present. The high, medium, and subdued factors for the eleven 
applicants were as follows, with the asterisks denoting the applicants 
who appeared in both the morning and afternoon sessions. 

Teacher Applicants' Characteristics 





High Factor 


!• 


originality 


*2. 


originality 


*3. 


systematic 


4. 


systematic. 


*5. 


empathy 


6. 


empathy 


7.— 


sociability 


*8. 


sociability 


*9. 


buoyancy 


10. 


buoyancy 


11. 


buoyancy 



Medium Factors 

buoyancy, sociability 
empathy, systematic 
sociability, empathy 
originality, buoyancy 
sociability, buoyancy 
systematic, originality 
empathy, originality 
buoyancy, systematic 
sociability, originality 
empathy, systematic 
originality, systematic 



Subdued Factors 

empathy, systematic 
buoyancy, sociability 
originality, buoyancy 
sociability, empathy 
originality, systematic 
sociability, buoyancy 
buoyancy, systematic 
empathy, originality 
empathy, systematic 
sociability, originality 
empathy, sociability 



Using this scheme as a basis, and being guided by the bipolar ad- 
jective scales describing these factors, a composite "personality" was 
developed for each of the eleven applicants. Scripts were then written 
with questions and answers designed to permit manifestation of the charac 
teristics of each applicant in clear and unambiguous manner. 
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APPENDIX E 

Examples of Single and Multiple Documents 

The first two pages of this appendix represent the single document 
used In the experiment. The first page was actually the upper half of 
a single folded page, while the second page was the lower half of the 
8V* X 22” single page. The remainder of the appendix Illustrates the 
application form used (three pages); placement credentials (four pages— 
general Information, student teaching record, undergraduate course record, 
letter of recommendation); and confidential teacher reference form used 
to acquire Information by telephone from references (one page). 



Wf 



SUMMARY OF APPLICANT'S QUALIFICATIONS 



GENERAL INFORMATION 



Name 



Age 



Ht. 



Position Desired 1. 
High School 



2 . 



3 . 



Colleges: BA at 



Graduate work at 



Certificate 



yr* 



degree, if any 



expiration dote 



Weight 



yr. 



Secondary School Activities prepared to direct 



Elementary School Activities prepared to direct: Piano Singing 



P*E. Other 

Languages 

Years of experience Level(s) 

Best Qualified to teach 

BACKGROUND DATA Benef iciol Possible Problem 

Considerable Somewhat None Somewhat Considerable 

Teach .g experience 

Youth activities 

Other work experience 



Professional membership and activities . . . . 

Hobbies related to teaching 

Honors and activities 

Travel 

Special knowledge, skills, abilities 



Marital status. . . . 

Children 

Spouse's occupation . 

Health 

Physical disabilities 
Draft status .... 
Criminal record . . . 
Habits or peculiarities 
Work absenteeism . . 



O 
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SUMMARY OF COLLEGE COURSEWORK 



! * 



Scholastic Average 4.00-3.30 3.29-2.90 

Mojo r 

Minor u 

Minor 

Professional Education 

Total Courses Token 

Breadth and Depth Above 

of Coursework ^ Exceptional Average 

Major 

Minor 

Minor 

Professional Education ^ 

General Education 



2.89-2.40 2.39-2.15 2.14-2.00 



Standard 



Very 

Minimum 



Deficient 



EVALUATIVE DATA 

The grid is a summary of the evaluations of the applicant's characteristics. 

Code: S-student teacher report, C-cooperating teacher recommendation, P-college professor 
recommendations, X-principal recommendation, Y-superintendent rec omme ndation 



CHARACTERISTIC 

Personal appearance 

Character 



HIGH 



LOW 
1 



Personal ity 

Responsibility ....... 

Emotional Stability 

Mature Judgment 

Adaptability 

Cheerfulness, sense of humor . 



Breadth of General Knowledge . . . 
Ability to present idea (written & oral) 
Ability to do academic work .... 
Attitude toward college work .... 



Knowledge of professional matters. . . . 

Knowledge of teaching methods 

Knowledge of subject matter & background 
Professional interest and/or growth . . . 
Interest in children (students) 



Teaching ability (overall) . . 

Classroom control 

Organization and management . 

Presentation 

Individualization 

Creativity and resourcefulness 



Cooperation 

Ability to work with others 



Recommended 
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An application for a teaching position in the 




Nnm» 


(Maiden) 


Date 




Zip 


Phone 


Permanent Address 


Zip 


Phone 




4. Date of Birth 5. Height 6. Weight 7. Citizen of 



8. Single Married Widowed Separated Divorced — 

9. If married, name of spouse 10. No. of Children Ages 

11. Spouse’s occupation 12. Spouse’s employer 

13. Condition of Health 

14. Have you any physical disabilities or limitations? 

15. Have you ever been arrested for other than traffic offenses? 

(If your answer to either of the last two questions was “Yes”, please explain fully below.) 




16. Position desired: 1) 2) 3) 

17. Why do you want to teach in this district? 



18. Type of certificate held 



State 



Expiration Date 



- 63 - 




NAME OF SCHOOL 


LOCATION 


DATES 

INCLUSIVE 


DEGREE & 
DATE REC'D 


MAJOR 


MINOR 


High School 
























College or 












University 
























Graduate or 












Special Work 


























DATES 



NO. OF 



SUBJECT {GRADES) 
TAUGHT OR POSITIONS 



FROM TO 



LOCATION 



NAME OF SCHOOL 



YEARS 



HELD 



REASON FOR LEAVING 





23« Credentials are on file at 



Address 




24. Special Qualifications: 

What student activities have you directed or coached? 1) 



2) 3) 4) 

What student activities are you prepared and willing to direct or coach? 1) 

2) 3) 4) 

25. Special Qualifications - Elementary (to be completed by elementary candidates only ) 

Can you play the piano? Can you direct class singing? Can you teach your own music class? 

Additional music classes? Can you teach your own art class? Additional art classes? 

Can you teach your own physical educational class? Additional Physical Education classes? 

Other , Foreign languages (Specify) 

26. List names of professional organizations to which you belong and/or expect to join. 

27. List hobbies or special abilities relating to teaching profession. 



28. Honors Received (High School^ College^ otherwise). 



29. College Activities (include participation in organizations). 

30. Travel (when^ where^ purpose) 



31. I hereby certify that the information herein is a true and complete statement of my personal and professional record 
to date. 
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Signature of Applicant 




GENERAL INFORMATION 

Name 

Position desired: Subject(s) 

Present address 

Home Address ■ 

Birth dote Height 

Marital Status Sex 

EDUCATION 

High School 

No. of 

College(s) Attended quarters 



CONFIDENTIAL Placement Credentials 
from 

Placement Office 
UNIVERSITY OF COLUMBIA 
Westland, Columbia 



Weight 

No. of Children 



Date _ 

Grades 

. Zip No. _ 

- Zip No. . 

- Health _ 

Ages 



^ Location 

Degree(s) Expected 

Major Minors or received 



Phone 

Phone 



Date 



PRACTICE TEACHING FIRST YEAR COMPETENCE 

Grade Subjects Place Qtr. Yr. 

Q Elementary 

□ Secondary 

Subject 

TEACHING EXPERIENCE OTHER WORK EXPERIENCE 

_ Length of 

Date District Name Subject or Grade Job Time 



Membership: Professional Organizations and Community Activities 



Travel 



Miscellaneous: (include experience related to teaching or children, military experience, etc.) 
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Name of Student Date of S/T 

S/T Assignment School 

District Name of Cooperating Teacher 

RATING ASSIGNED 

Rate by checking (y') the appropriate column: 

(Where satisfactory is achieved by the 
average student) 

Factor Rated: 

I. PROFESSIONAL QUALITIES 

1. Enthusiasm for teaching 

2. Standard of ethical behavior 

3. Accepts responsibility 

4. Adjusts to new situations 

5. Openness to ideas and new fields of knowledge. 

II. PERSONAL QUALITIES 

1. Dress and appearance 

2. Poise and manner 

3. Health and vitality 

4. Emotional maturity and adjustment 

5. Gets along well with others 

6. Respected by pupils 

7. Ability in realistic self-evaluation 

8. Speech and voice 

III. KNOWLEDGE AND PREPARATION 

1. Subject-matter and background 

2. General education background 

3. Exhibits creativity and initiative 

4. Oral communication skills 

5. Written communication skills 

IV. TEACHING PERFORMANCE 

1. Creates varied, stimulating learning situations 

2. Shows genuine respect for all students 

3. Gives clear, concise directions 

4. Displays fairness end sound judgment 

5. Maintains reasonable standards of evaluation 

6. Provides for individual differences 

7. Organization for and management of small- 
group work 

8. Differentiates assignments 

9. Utilizes problem solving techniques 

V. CLASSROOM MANAGEMENT 

1. Efficiency in use of time, materials, plant 

2. Records and reports 

3. Classroom control 

4. Promotes and secures cooperative behavior 

5. Daily lesson planning 

6. Long range unit plan 

VI. COMPOSITE EVALUATION 

VII. SUMMARY STATEMENT (May include attendance, restrictions, handling of confidential materials, extra- 
curricular participation, etc.) 





Comments on particular strength 
‘ areas needing improvement: 



RECOMMENDED FOR THE FOLLOWING LEVEL: 



VIII. 



Candidate’s Name 



Nature of association with person reporting 



Date 






Signature of person reporting 

Position or Title 

INSTITUTION 
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NORWEST COMMUNITY SCHOOLS 

P. 0. 134 
Norwest, Columbia 

Confidontiai Toachor Roforonco 



Dote. 



teacher of. 



has applied for a position in this District as a 

Will you kindly give your frank evaluation of the professional 



«.. . ti 7~, : * •''no'y give your rronk evaluation of the orofej 

and personal eharaetan sties of ths applicant? Yaur eanfidene. will be respected and ji^jreooperatian appreerated. 

Sincerely yours, 



Rating Standards: 

Superior - -- -- -- -- Equal to top ten per cent of your teachers 

Above average but not excellent 

^''f®9® Satisfactory but not at all outstanding 

Below Average- - - - - Below average but fair 
Unacceptable ----- Definitely unsatisfactory 

le TEACHING ABILITY - To what extent has this person the 
makings of a master teocher in subject matter and methods? 

2. STUDENT CONTROL -- To what extent is this person able to 
maintain constructive discipline through purposeful activity 
rather than repression or disorder? 

3. PERSONAL APPEARANCE - To what extent does this person 
appear healthy and have good habits in grooming and dress? 

4. CHARACTER - To what extent does this person exhibit whole- 
someness in character and personal values? 

5. EMOTIONAL STABILITY - To what extent does this person 
maintain poise and good nature in spite of the irritations of the 
job and personal troubles? 

6. INTEREST IN STUDENTS — To what extent is this person in- 
terested in students and their activities in and out of the class- 
room? 

7. PROFESSIONAL INTERESTS AND GROWTH - To what extent 
is this person interested in teaching as a career, and have the 
willingness and ability to improve? 

8. COOPERATION — To what extent does this person work har- 
moniously and beyond the line of duty with the administration, 
fellow teachers, school and community? 

QUESTIONS: 

How long have you known the applicant? 

In your opinion, this person is best qualified to teach 

I I. . . ... 



Lane R, Holmes, Superintendent 




o 

// 


4 #/ 


7J 


^ / 


/ 





In what capacity? 



! , ..... »|wuiiii«u lo Tcgtjn 



, ■■■ ■ ,.i II m 

About how mony days obsonco hos this person ovsrogsd per yeor in your emolovmant’ 

Would you re-employ? Eogerly?' Look first? Only in emergency? 

Comments: 
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APPENDIX F 



Examples of Each Part of the Subject Response Form 







'■ 



f 



SUBJECT RESPONSE FORM 



For the Experiment 



VARIABLES AFFECTING DECISION MAKING 
IN THE SELECTION OF TEACHERS 






Conducted by 

Dr. Dale L. Bolton 
Associate Professor of 
Educational Administration 
University of Washington 
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L. 



Subject Number 


(1-3) 


Date 


(4-5) 


Group 


(6-7) 


Session 


(8) 



Time Started ^ 
Time Finished 



(9-11) 




CLASSROOM OBSERVATION RECORD 
NORWEST COMMUNITY SCHOOLS 
NORWEST, COLUMBIA 



Everyone appraises - with or without a formal program. All the more reason why a systematic method is needed. 

The observation record is simply on attempt to think clearly about each teacher’s performance and potential against 
the background of his total work situation. Its purpose is to improve student instruction through encouraging in- 
service growth and systematizing supervisory counseling. 

For a given observable characteristic, bi-polar terms are used to describe opposite qualities. The line representing 
the continuum from one extreme to the other is divided into seven segments. The mid-segment on the scale denotes 
“neutral" qualities pertaining to the characteristic. 

Three different ratings on the continuum calm-excitable are given for examples: 



Calm JL : : I I : : Excitable 



A check In this category indicates the person rated is extremely calm. He exhibits freedom from useless 
agitation in all situations. 



Calm 



Excitable 



A check in this category indicates the person rated has no pre-disposition toward either calmness or 
excitability. Whatever the situation or concern, he is as likely to exhibit composure and act logically as 
he is to show useless agitation and abandonment of logical methods. 



Calm 



I X : : Excitable 



A check In this category indicates the person rated exhibits a tendency toward excitability. When ex- 
changing ideas, planning, or working with others, he often displays a degree of agitation which somewhat 
impairs his composure and logical functioning. However, he seldom, if ever, shows uncontrolled agitation. 

For each teacher that you observe, place a check (X) in the appropriate spaces on the observation instrument. 
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Ttachtr 






( 12 - 13 ) 



A. PLANNING & ORGANIZING CLASSWORK 



1. Purposeful 
2* Systemotic 

3. Cooporotivo 

4. Orifinol 



Aimless 

Disorganized 

Antagonistic 

Unimaginative 




Comments 



B. CLASSROOM MANAGEMENT 



1. ' Punctuol 

2. Controlled 

3. Consistent 

4. Flexible 

5. Foir 

6. Responsive 

Cornmsnts. 





— • : Tardy 

- : : Disorderly 

: : inconsistent 

: : Fixed 

: •• Partial 

• •• Indifferent 



C. CREATING A MOTIVATIONAL ENVIRONMENT 




( 18 ) 

( 19 ) 

( 20 ) 
21 ) 

( 22 ) 

( 23 ) 



1. Steady 

2. Flexible 

3> Bread*minded 

4. Sensitive 

5. Kindly 

6. Sense of humor 
> 7. Stimulating 

Comments 

D. INSTRUCTION 

1. Alert 

2. Resourceful 

3. Poised 

4. Helpful 

5. Inspirrtional 

6. Precise Comm. 

7. Pleasant 

Comments 

E. EVALUATION 

1. Continuous 

2. Rational 

3. Systemotic 

4. Just 

Comments 




Spasmodic 

Rigid 

Narrow>minded 

Unfeeling 

Critical 

Humorless 

Dull 





Apathetic 

Inflexible 

Agitable 

Hindering 

Uninspiring 

Fuzzy 

Harsh 





Erratic 

Irrational 

Disorganized 

Inequitable 



( 38 ) 

39 ) 

l»0) 

(M) 



F. GUIDANCE AND COUNSELING 



* 1. Resourceful 

2. Patient 

3. Approachable 

4. Inspiring 

5. Communicative 

Comments 


• e 1 


1 1 
1 1 
1 1 
1 1 


■■ 


Trite ^42) 

Impatient (43) 

: Formal (44) 

: Uninspiring (45) 

: Uncommunicative (46) 


G. OUT-OF-THE-CLASSROOM DL 

1. Active 

2. Responsible 

3. Skillful 

4. Accurate 

5. Punctual 

6. Constructive 

7. Progressive 

Comments 


ITIES Al 


SID PROFESSIONAL AC 

1 1 

1 1 

1 1 

1 1 

1 1 


nVITIE! 


5 

Evading (47) 

Irresponsible (48) 

Unskillful (49) 

Inexact (50 ) 

Tardy (51) 

Antagonistic (52) 

Stagnant (55) 


.. H. RELATIONS WITH OTHER PE 

1 . Approachable 

^ 2. Cooperative 

3. Discreet 

4. Responsible 

5. Open*mindecl 

6. Effective 

Comments 


:OPLE - 


- (STAFF AND PAREN' 

1 1 

1 1 

1 ! 

1 1 


rs) 


Aloof (54) 

Uncooperative (55) 

Imprudent (56) 

Irresponsible (57) 

Narrow-minded (58) 

ineffective (59) 


1. SCHOOL-COMMUNITY RELATI 

1. Active 

2. Initiates : 

3. Effective : 

Comments 


ONS 


1 1 : 

1 1 

1 1 




Inactive (60) 

Follows (61) 

Ineffective (62) 
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Degree of Certainty: Teacher Evaluation Instrument 



Tou have predicted how each applicant would he rated on the 
Teacher Evaluation Instrument at the end of his first year of teaching. 
Suppose that you were given what is to you a "fair amount" of money 
which you could either keep or het any portion of on the "correctness" of 
your predictions for the applicants. How much would you he willing to 
het that your predictions for the applicants are generally "correct? " 



0^ 1 (63) 

10 ^ 2 

25 ^ 3 

50 ^ 4 

5 

90 ^ 6 

100^ 7 



- 77 - 






Ranking of the Applicants 



Rank the eight applicants in the order of their desirability for 
the fourth grade teaching position in Norwood Elementary School, Norwest 
Community School District. 



Rank 

1 . 

2 . 

3. 

k. 

5. 

6 . 

7. 

8 . 



Last Name of Applicant 



(64) 

(65) 

( 66 ) 

(67) 

( 68 ) 

(69) 

(70) 

(71) 



Degree of Certainty: Ranking 



Suppose you were given what is to you a "fair amount" of money 
which you could either keep or het any portion of on the "correctness" of 
your decisions regarding teacher selection. How much would you he willing 
to het that your rank ordering of the eight applicants is "correct," 
except perhaps for one or two "errors" of slight misplacement? 



Oft 1 

10?6 2 

25 ^ 3 

5056 k 

75 ^ 5 

90ff> 6 

100 ^ 7 
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Fineness of Discrimination 



On some characteristics certain people are more alike than dissimilar. 

On other characteristics^ these same people may be quite dissimilar. 

When they are more alike than unlike, we often classify or group them 
into homogeneous groups. 

Below five characteristics are listed. For each characteristic use the 
nuojbers of the applicants indicated below to show how many groups you 
consider to exist in the eight teacher applicants you have Just examined 
and what individuals '^belong" to each group. 

For example, if for a given characteristic you believe that applicant 
is more unlike than like the others; that applicants 1,5,6,8 are more 
s imi la r than dissimilar; and that 2,3,T likewise for a homogeneous group, 
then you should show this by writing (4). (l«5«6.8). (2,3,7) 



Characteristic Groups 



1. 


Adaptable (imaginative and resourceful) 


(73) 


2. 


Shipathetlc (warm and understanding) 


(ru) 


3. 


Systematic (responsible and definite) 


(75) 


k. 


Buoyant (stimulating and expressive) 


(76) 


5. 


Probable success in teaching 


(77) 



1. Connie Andrews 

2. Sharon Barnes 
3* Tami Beck 

i|-. Melody Cochran 
5* Janet Lee 
6. Baula Lewis 
7 • Sally Mickelson 
8. Marlene West 
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APPENDIX G 

Reactions of the Subjects to the Experiment 

At the end of each day's sessions, meetings were held with the sub- 
jects to obtain their reactions to the experimental task and to gain 
insights regarding the simulated situation. The purpose of acquiring 
this information was to assist in determining the feasibility of using 
the developed materials for training of educational administrators and 
for conducting additional research regarding decision making in the 
selection of teachers. (Another function of these sessions was to ask 
the subjects' cooperation in not revealing information to other admini- 
strators who would be participating in the experiment.) 

The more frequent reactions and concerns expressed by the subjects 

were : 

1. The task was demanding because of the perceived similarity 
of teacher applicants. 

2. The time allocated to the morning session was a pressing 
aspect in completing the experimental task. 

3. Making estimates of applicant's performance and completing other 
responses was difficult because subjects were instructed to 
review all eight applicants before responding to the various 
measurement Instruments. 

4. There was some difficulty in assuming the roles of an admini- 
strator in the hypothetical situation throughout the entire 
experiment, especially in the afternoon session. 

5. There was considerable realism of the simulated materials used 
in the orientation and experimental task. 

6. Masking of information caused some frustrations in that some 
subjects had difficulty making decisions without complete 
information. 

7. The materials used in the experiment would be beneficial in 
the training of educational administrators or in the continu- 
ing education of principals. 



